Avalable onlne at www.scencedrect.com Proceda Engneerng 41 (2012 ) 156 162 Internatonal Symposum on Robotcs and Intellgent Sensors 2012 (IRIS 2012) Multple Robots Formaton A Multobctve Evoluton Approach Genc Cap* and Zulkfl Mohamed Unversty of Toyama, Gofuku 3190, Toyama, Japan Abstract In ths paper, we present a new method for multple robots formaton, whch means certan geometrcal constrans on the relatve postons and orentatons of the robots throughout ther travel. In our method, we apply multobectve evolutonary computaton to generate the neural networks that control the robots to get to the target poston relatve to the leader robot. The advantage of the proposed algorthm s that n a sngle run of multobectve evoluton are generated multple neural controllers. We can select neural networks that control each robot to get to the target poston relatve to the leader robot. In addton, the robots can swtch between neural controllers, therefore creatng dfferent geometrcal formatons. The smulaton and expermental results show that the multobectve-based evolutonary method can be appled effectvely for generatng neural networks whch enable the robots to perform formaton tasks. Keywords: Robot formaton, neural networks, evoluton. 1. Introducton Recently, a lot of research s conducted n teamng and cooperaton of multple robots. One mportant research ssue n systems of multple robots s the moton plannng for formaton paradgm. Several approaches have been proposed to address the problem of multple robots formaton. The approaches range from leader followng [1], [2], vrtual structures [3], [4] and vrtual leaders [5], [6]. In other works, socal potentals [7] and formaton constraned functons [8] are used to gude robots nto formatons. Neural networks have been also appled to mantanng formatons. In [9] a vson-based movng n formaton by four moble robots was presented. One robot, the leader, goes frst provdng movng plans to the other robots who follow the leadng robot. In moton control, for each robot a radal bass functon network approxmated by learnng s used. Ths artcle presents a novel approach for multple robot formaton based on multobectve evolutonary algorthms (MOEAs) [10]-[12]. Unlke prevous methods, n the experments presented here, the accumulated error between the target and real poston of each robot relatve to the leader robot s consdered as a separate obectve functon. The nondomnated sortng genetc algorthm (NSGA II) [13] s used to generate the Pareto set of neural networks that tradeoff between the separate task performance. MOEAs have been successfully appled to evolve neural networks n whch the archtectural complexty and performance are co-optmzed [14]. MOEAs have also been appled to desgn feedforward neural networks for multple task performance [15] and tme seres forecastng. In addton, Barlow et al. [16] employed the multobectve genetc programmng to evolve controllers for unmanned aeral vehcles. In ths paper, the MOEA s appled for the frst tme to evolve neural controllers for multple robots formaton. The specfc questons we ask n ths study are whether MOEAs can successfully generate neural controllers for general stable multple robots formaton tasks; f the evolved neural controllers can be appled for dynamc swtchng between dfferent formatons. In order to answer these questons, n the experments reported here, we consder the evoluton of neural controllers for e-puck robots that have to follow the leader robot n specfc relatve postons. * Correspondng author. Tel. : 0081-76-445-6745 E-mal address: cap@eng.u-toyama.ac.p 1877-7058 2012 Publshed by Elsever Ltd. do:10.1016/.proeng.2012.07.156
Genc Cap and Zulkfl Mohamed / Proceda Engneerng 41 ( 2012 ) 156 162 157 In order to further verfy the effectveness of the proposed method for multple robots formaton, the number of robots s ncreased. In the proposed method, we evolve the weght connectons of the neural controller for each robot. As the number of robots n the formaton ncreases some robots poston do not match wth the target ones. Ths s because the target postons spreads n a wde area. Therefore, the ntal poston of the robot durng evoluton nfluences the ftness functon. Ths makes the evoluton process dffcult. The robustness of the neural controllers evolved n the smulaton was also tested n the hardware experments. 2. Multobectve Evolutonary Algorthm In multobectve optmzaton problems there are many (possbly conflctng) obectves to be optmzed, smultaneously. Therefore, there s no longer a sngle optmal soluton but rather a whole set of possble solutons of equvalent qualty. Consder wthout loss of generalty the followng multobectve maxmzaton problem wth m decson varables, x parameters and n obectves: y f ( x) ( f 1( x1,... xm),..., (1) f n( x1,... xm)) where x ( x1,... xm ) X, y ( y1,... yn ) Y and where x s called decson parameter vector, X parameter space, y obectve vector and Y obectve space. A decson vector a X s sad to domnate a decson vector b X (also wrtten as a b) f and only f: {1,..., n}: f ( a) f ( b) (2) {1,..., n}: f ( a) f ( b) NSGAII was employed to evolve the neural controller where the weght connectons are encoded as real numbers. In [17], the authors compared the NSGAII wth four other multobectve evolutonary algorthms usng two test problems. The NSGAII performed better than the others dd, showng that t can be successfully used to fnd multple Pareto-optmal solutons. In NSGAII, before selecton s performed, the populaton s ranked on the bass of domnaton usng Pareto rankng. 3. Multobectve Evolutonary Algorthm 3.1. Formaton task The developed system s shown n Fg. 1(a). The robots have to get to ther poston relatve to the leader robot and keep the geometrcal formaton throughout ther travel. Because the robots ntal postons are dfferent from the target ones, the robots have to move fast to reach to ther poston relatve to the leader robot. The entre envronment s a rectangle of surrounded by walls. The ndvdual lfe tme of each robot s 500 tme steps, where each tme step lasts 0.1s. Durng ths tme the leader robot moves wth a constant velocty of 0.1m/s. The frst task conssts of two robots followng the leader robot formng a trangle of a predetermned shape. (a) (b) Fg. 1. Formaton task: (a) Developed system; (b) Formaton task.
158 Genc Cap and Zulkfl Mohamed / Proceda Engneerng 41 ( 2012 ) 156 162 3.2. Neural Archtecture We mplemented a smple feed-forward neural controller wth 3, 2 and 2 unts n the nput, hdden and output layers, respectvely. The reason that we selected a smple robot s that we are nterested to conduct evoluton of neural controllers n the real hardware. The nputs of the neural controller are the angle (A Leader ) and dstance (D Leader ) of the E-puck robot relatve to the leader robot and orentaton of the E-puck robot (Fg. 1(b)). The egocentrc angle to the leader robot vares from 0 to 1 where 0 corresponds to 45 o to the rght and 1 s 45 o to the left of the leader robot. Random nose, unformly dstrbuted n the range of +/- 5% of sensor readngs, has been added to the angle of the E-puck robot. Because the dstance to the E-puck robot durng the experments s determned based on the number of pxels, the random nose n smulatons s consdered n the range of +/- 10%. The hdden and output unts use sgmod actvaton functon: 1 y (3) 1 x e where the ncomng actvaton for node s: x w y (4) and ranges over nodes wth weghts nto node. The output unts drectly control the rght and left wheel angular veloctes where 0 corresponds to no moton and 1 corresponds to full-speed forward rotaton. The maxmum forward velocty s consdered to be 0.1 m/s. Robot 1 Robot 2 Fg. 2. Nondomnated optmal solutons of dfferent generatons. Fg. 3. Smulaton and expermental results (2 robots).
Genc Cap and Zulkfl Mohamed / Proceda Engneerng 41 ( 2012 ) 156 162 159 3.3. Evoluton For any evolutonary computaton technque, a chromosome representaton s needed to descrbe each ndvdual n the populaton. The genome of every ndvdual of the populaton encodes the weght connectons of the neural controller. The connecton weghts range from -5 to 5. The target dstance d t between the leader and E-puck robots s consdered 0.3m, whle the target angle +/-15 degree. Durng evoluton each neural of the populaton controls a sngle robot moton and at the end of ts lfetme the ftness functon for each target poston s calculated. In order to mnmze the dfference between the target and real poston relatve to the leader robot, the ftness f of each robot n the formaton (=1~number of robots n the formaton), s calculated as follows: f max_ st 1 ( d d t r ) where max_st s the maxmum number of steps, d r and d t are the real and the target dstance, and ɵ r and ɵ t are the real and target angle. If an ndvdual happens to get out of the vsual feld, ht the leader robot or the wall, the tral s termnated and a low ftness s assgned. Therefore, such ndvduals wll have a low probablty to survve. The followng genetc parameters are used: N ger =30, N pop =100, σ shared =0.4. 4. Results All the robots have a wreless communcaton wth the control PC. The leader robot s equpped wth a wreless camera used to calculate the dstance, angle and drecton of the robots. We frst dscuss the best solutons obtaned from the MOEA, for a smple formaton mechansm where two robots have to be postoned relatve to the leader robot. Fg. 4 shows the nondomnated optmal front for generatons 1, 10, and 30, averaged for fve dfferent runs of MOEA. Durng the frst 30 generatons there s a great mprovement of the qualty and dstrbuton of nondomnated optmal solutons. The nondomnated optmal front of generaton 30 has a clear tradeoff between the two obectve functons (Fg. 2). The extreme solutons represent the best Neural Networks that control each robot to get to the target poston relatve to the leader robot and keep ths throughout the moton. Smulaton and expermental results where the E-puck robots form a trangle wth the leader robot are shown n Fg. 3. Intally, the E-puck robots poston s dfferent from the target ones. Therefore, the robots move quckly to get nto the desred poston relatve to the leader robot. t r (5) (a) Hnton dagram. (b) Neuron actvaton Fg. 4. Robot 1 neural controller.
160 Genc Cap and Zulkfl Mohamed / Proceda Engneerng 41 ( 2012 ) 156 162 (a) Hnton dagram. Fg. 5. Robot 2 neural controller. (b) Neuron actvaton Fg. 6. Formaton task wth three robots. The Hnton dagram of Robot 1 NN weght connectons (Fg. 4(a)) show that ntally, the hdden unt 1 (H1) s fully actvated. Due to postve connecton wth the LeftMotor unt and negatve connecton wth the RghtMotor unt, the LeftMotor unt s nearly fully actvated and the RghtMotor unt s deactvated (Fg. 4(b)). Therefore, the Robot 1 frst rotates clockwse to reach the target poston relatve to the leader robot. Intally, the Robot 1 s postoned n the front of the leader robot (ang=0.5) and as the robot moves the angle converge to the target one. When the robot drecton s not headed toward the leader robot (Dr=-1), the Robot 1 makes a quck rght turn. Then t follows the leader robot keepng the relatve poston. The Robot 2, ntally s not headed toward the leader robot (Dr=-1). Due to a strong negatve connecton between the H 2 wth the Left wheel unt, as shown n Fg. 5(a), the left wheel essentally stops movng whle the rght wheel contnues to rotate wth nearly the maxmum velocty (Fg. 5(b)). Therefore, the Robot 2 rotates counterclockwse to move to the target poston. Then the Robot 2 strategy s to move slowly clockwse untl the leader robot s not n front of the robot. Due to the dscontnuty n actvaton of Dr ecton, the E-puck robot turns quckly conterclockwse (the L wheel unt s fully actvated).
Genc Cap and Zulkfl Mohamed / Proceda Engneerng 41 ( 2012 ) 156 162 161 Fg. 7. Dfferent formatons by swtchng between neural controllers. We mplemented the evolved optmal neural controllers on the real hardware of the E-puck robots. The dstance and angle of the E-puck robots are calculated usng the vsual sensor. Because the dstance s calculated based on the number of pxels, whle the angle based on the blob poston, there are some dfferences between the smulated and real robot performance. For example, the dstance and angle nput unts have some dscontnuty n ther actvaton. However, despte these dfferences, the robots stll performed the formaton task well. An mportant advantage of applyng MOEAs to evolve neural controllers for multple robot formaton s the ease wth whch the number of robots may be ncreased. In the followng, we present the results where another robot target poston s added n front of the target robot. However, the robot postons are not the same wth the target ones (Fg. 6). For example, Robot 3, after reachng the target poston, contnues to move forward. As the number of robots n the formaton ncreases some robots poston do not match wth the target ones. Ths s because the target postons spreads n a wde area. Therefore, the ntal poston of the robot durng evoluton nfluences the ftness functon. For example, even f good ndvdual (NN) of the populaton can get a bad ftness f the dstance between the ntal and the target poston s large and vce-versa. Ths problem can be solved by ncreasng the number of robots durng evoluton consderng as ftness functon the average ftness of all robots. The ntal postons of multple robots can be desgned based on the target postons, reducng the effect of the ntal and target postons n the ftness functon. Smultaneous evoluton of multple neural networks controllng the robots to reach the target poston relatve to the leader robot, makes t possble that robots swtch between neural controllers resultng n dfferent formatons. Fg. 7 shows two robots swtchng among three evolved neural controllers. Frst, two robots are controlled by the neural networks that control them to get to the target 1 and target 2 poston relatve to the leader robot. Then, whle the Robot 1 contnues to be controlled by the same NN, the Robot 2 swtches to another neural controller. Therefore, the Robot 2 moves to another target poston relatve to the leader robot. 5. Conclusons Ths paper has expermentally nvestgated the effectveness of applyng MOEAs to address the multple robot formaton problem. In partcular, t was demonstrated that n a sngle run of the MOEA, robust neural controllers for each robot to get to the target poston relatve to the leader robot and keep t throughout the moton. The robustness of evolved neural controllers was also tested on the real hardware. For future work, we wll address two possble extensons. Frst, we plan to ncrease the number of robots durng evoluton consderng as ftness functon the average ftness of all robots. The second possble extenson s to develop neural controllers for formaton task n more dynamc envronments. References [1] Parker, L. E., 1998. Allance: An archtecture for fault tolerant multrobot cooperaton, IEEE Transactons on Robotcs and Automaton, 14/2, pp. 220-40,. [2] Desa, J. P., Ostrowsk, J. P. Kumar, V., 2001. Modelng and control of formatons of nonholonomc moble robots, IEEE Transactons on Robotcs and Automaton, 17/6, pp. 905-908,.
162 Genc Cap and Zulkfl Mohamed / Proceda Engneerng 41 ( 2012 ) 156 162 [3] Beard, R. W., Lawton, J., Hadaegh, F. Y., 2001, A feedback archtecture for formaton control, IEEE Transactons on Control Systems Technology, 9/6, pp. 777-790. [4] Lews, M. A., Tan, K., 1997. Hgh precson formaton control of moble robots usng vrtual structures, Autonomous Robots, 4, pp. 387-403,. [5] Leonard, N. E., Forell, E., 2001. Vrtual leaders, artfcal potentals and coordnated control of groups," n Proceedngs of the IEEE Conference on Decson and Control, Orlando, Florda, pp. 2968-2973. [6] P. Ögren, E. Forell, and N. E. Leonard, 2002. Formatons wth a msson: Stable coordnaton of vehcle group maneuvers, Proceedngs of 15th Internatonal Symposum on Mathematcal Theory of Networks and Systems. [7] Balch, T., Hybnette, M., 2000. Socal Potentals for Scalable Mult Robot Formatons, Proc. IEEE Internatonal Conference on Robotcs and Automaton, pp. 73 80. [8] Egerstedt M., Hu, X., 2001. Formaton constraned mult agent control, IEEE Transactons on Robotcs and Automaton, 17/ 6, pp. 947 951. [9] Hrota, K., Kuwabara, T., Ishda, K., Myanohara, A., Ohdach, H., Ohsawa, T., Takeuch, W., Yubazak, N., Ohtan, M., 1995. Robots movng n formaton by usng neural network and radal bass functons, Proceedngs of the 1995 Internatonal Conference on Fuzzy Systems, vol. 5, pp. 91 94. [10] Coello, C. A., Van Veldhuzen, D. A., Lamont, G. B., 2002. Evolutonary Algorthms for Solvng Mult-Obectve Problems, Kluwer Academc Publshers, New York. [11] Deb, K., 2001. Mult-Obectve Optmzaton usng Evolutonary Algorthms, John Wley & Sons, Chchester, UK,. [12] Fonseca, C.M., Flemng, P.J., 1995. An overvew of evolutonary algorthms n multobectve optmzaton, Evolutonary Computaton, 3, pp. 1-16,. [13] Tran, K. D., 2009. An Improved Non-domnated Sortng Genetc Algorthm-II (ANSGA-II) wth adaptable parameters, Intl. Jour. of Intellgent Systems Technologes and Applcatons, 7/ 4, pp. 347-369. [14] Abbass, H. A., 2001. A Memetc Pareto Evolutonary Approach to Artfcal Neural Networks, The Australan Jont Conf. on Artfcal Intellgence, Adelade, Lecture Notes n Artfcal Intellgence LNAI 2256, Sprnger-Verlag, 1-12,. [15] Cap, G., 2007. Multobectve Evoluton of Neural Controllers and Task Complexty, IEEE Transactons on Robotcs, 23/6, 1225-1234. [17] Barlow, G. J., C. Oh, K., Grant, E., 2004. "Incremental evoluton of autonomous controllers for unmanned aeral vehcles usng mult-obectve genetc programmng. Proceedngs. of IEEE Int. Conf. on Cybernetcs and Intellgent Systems (CIS). [18] Das A. H. F., De Vasconcelos, J. A., 2002. Multobectve genetc algorthms appled to solve optmzaton problems, IEEE Transactons on Magnetc, 38/2, pp. 1133-1136.