Parameter Optimisation of an Evolutionary Algorithm for On-line Gait Generation of Quadruped Robots

Transcription

1 Parameter Optmaton of an Evolutonary Algorthm for On-lne Gat Generaton of Quadruped Robot Drago Golubovc and Huoheng Hu Department of Computer cence, Unverty of Eex Colcheter CO Q, Unted Kngdom Emal: Abtract Th paper preent a hybrd evolutonary algorthm (EA) for developng locomoton gat of ony legged robot. An onlne tranng algorthm ued for generatng gat for quadruped walkng robot baed on a hybrd approach that change the probablty of genetc operator n repect to the performance of the operator' offprng. The probablty of applyng an operator change n proporton to the oberved performance of the ndvdual created by that operator n the coure of a run. The electon of EA parameter uch a the populaton ze and recombnaton method and mutaton parameter are made to be flexble and trve toward optmal performance autonomouly. An overhead CCD camera ued to evaluate the performance of the generated gat on-lne whle the robot playng a football game. Robot learnng to walk on t own wthout any human nterference. Introducton Legged robot how gnfcant advantage over wheeled robot nce they can travere n uneven and untructured envronment. Th hgh moblty tem from the fact that n contrat to wheeled or tracked robot legged robot dcretely contact wth ther envronment va ther leg. Th, however, nevtably caue hghly complcated dynamc between the robot and ther envronment. On the other hand wheeled robot can acheve much greater peed and moblty on flat ground [6]. We took advantage of th oberved property of wheeled robot and we generated walkng moton for the robot that mmc the movement of wheel but tll we tred to preerve flexblty and adaptablty of legged robot (fg. ). By placng ome mnor retrcton on how the leg moton are generated, we develop an ealy computable clafcaton of the bet gat pattern to be ued for any gven moton (forward, backward, deway or rotatonal) of ony AIBO robot. The objectve of the preent work to make effcent evolutonary algorthm (EA) eaer n order to olve large ntance of dffcult combnatoral optmzaton problem wthn an acceptable amount of tme. No known technque allow one to exactly olve wthn an acceptable amount of tme, uch dffcult combnatoral optmzaton problem. Moreover, tradtonal approache that are ued to fnd ub-optmal oluton are not alway atfactory nce they are ealy attracted by local optma. Evolutonary algorthm (EA) that are npred by natural evoluton mechanm explore dfferent regon of the earch pace concurrently. They are thu often trapped n a local optmum and are not well uted to treat dffcult combnatoral optmzaton problem. They are alo greedy n computaton power and memory pace. Thu, t n general extremely dffcult to degn gat controller for legged robot. o far varou method have been propoed to contruct legged-robot gat controller, whlt very few tude have nvetgated the degn of controller conderng the nteracton dynamc wth the envronment [][][5]. However, nce the above approache are baed on a hand-crafted manner, t quetonable whether or not thee approache wll be tll feable (.e. ealy mplemented) a the complexty of the dered tak and the nteracton dynamc ncreae. On the other hand, recently the Evolutonary Robotc (ER) approach ha been attractng a lot of concern n the feld of robotc and artfcal lfe [][]. In contrat to the conventonal approache where degner have to contruct controller n a top-down manner, the method n the ER approach have gnfcant advantage nce they can autonomouly and effcently contruct controller by takng embodment (e.g. phycal ze and hape of robot, enor/motor properte and dpoton etc.) and the nteracton dynamc between the robot and t envronment nto account. Fg. Paw trajectory Runnng a genetc algorthm ental ettng a number of parameter value. Fndng ettng that work well on a problem a non-trval tak. If poor ettng are ued, the performance of a genetc algorthm can be everely mpacted. We propoed a technque for ettng the probablte of applyng genetc operator durng the coure of a run. The technque nvolve adaptng the operator probablte baed on ther oberved performance a the run take place.

2 Two ueful GA tratege were employed by reearcher to fnd good operator probablte. One wa contaned n DeJong' the work [9] and the other wa decrbed by Grefentette n [0]. The problem DeJong and Grefentette were attackng wa that of determnng robut parameter ettng for populaton ze, operator probablte, and evaluaton normalaton technque. The problem of adaptng genetc algorthm parameter n general wa mplfed, n that the only chromoomal repreentaton technque ued wa the bt trng, and only two operator were condered - mutaton and croover. The ret of th paper organed a follow. ecton decrbe the fundamental ue for a walkng robot that tatcally and dynamcally table. ecton preent the genetc modellng of wheeled moton for ony AIBO robot. The evoluton mechanm addreed n ecton. Then, ecton 5 nvetgate the ue of mult-objectve optmaton algorthm. Expermental reult are preented n ecton 6 to demontrate the feablty. Fnally, a bref concluon and future work are gven n ecton 7. tatcally and Dynamcally table Walkng Gat A gat a cyclc, perodc moton of the jont of a walkng robot. For quadruped, tatcally table walkng gat have at leat, and dynamcally table gat at leat leg, n contact wth the ground at all tme [8]. The leg of the robot cycle through the gat wth a partcular phang. We preent a method to chooe the phang accordng to the drecton of moton of the centre of ma of the robot. A et of requrement were drawn up that the walkng needed to atfy. Thee ncluded: Omn-drectonal moton - The robot hould be able to walk n all drecton. The gat hould move the robot a fat a poble. nce the robot would be movng forward mot of the tme, t wa decded that the robot hould be able to move forward at the maxmum peed poble. Phae Phae Phae Phae Fg. Foot placement curve Fg. Walk forward tatcally table ¾ duty cycle provde a table gat for at leat one type of dered moton. Only n phae a leg not n contact wth the ground and we are tryng to make tatcally table gat we concluded that each of the four leg hould be n the dfferent phae. Leg ha to follow the equence of phae --- (fg. ). That leave u wth 6 poble phang of the leg of quadruped walkng ung ¾ duty cycle. A L Fg. 5 de walk V X ω V Y C B V x and V y - are tranlaton velocte ω - rotatonal velocty c center of gravty L tep length Fg. Robot Top Vew tablty - It wa mportant that the robot hould reman table at all tme. The method hould be computatonally fat and mplementable n real-tme. Computatonal effcency wa mportant becaue of lmtaton of the peed of the onboard proceor and avalable memory. Fg. 6 Rotaton to the rght Note that (V x, V y, ω) decrbe movement of the robot body. To mplfy problem let aume that only one of thee velocte ha potve or negatve value whle the other two are zero. In that cae we can fnd at leat one

3 tatcally table phang (among 6 poble) for forward, backward, left, rght and rotatonal movement of the robot. The choce of phang that maxmze tablty margn gven n the followng table: Leg Front-left Hnd-left Hnd-rght Front-rght Forward pha. de walk Rotaton Tab. Leg Phang Each phae done n ¼ of a tep cycle T. Accordng to th phang we can fnd out maxmum tranlatonal and angular velocte that can be acheved (equaton -) L V x =, for V y =0 and ω =0 () L V y =, for V x =0 and ω =0 () L arctg ( ) a + b ω, for V x =0 and V y =0 () L V x = coα, L V nα y =, for ω =0 () Dynamc walkng gat are fater than tatcally table gat. However, they are harder to control and mplement. We preent here detal of the expermental mplementaton of dynamc walkng for a quadruped. Dynamc walkng mlar to that decrbed earler for tatc walkng. It nvolve generaton of foot placement curve that are a functon of the drecton of moton of the robot. Invere knematc then ued to calculate the jont moton requred to generate the foot placement curve for the partcular gat. Maxmum duty cycle that can be acheved through dynamc walk ½. dc duty cycle ( ¾, ½ ) T P = T P =(-dc)t, T P = T P =(dc-½)t, for dc ( ¾, ½ ) FL HL HR FR T Perod o f tatc ntablty Forward walk Fg. 7 Perod of tatc ntablty Equaton 5 preent perod of tme n whch the robot tatcally ntable regardng duty factor for forward, de an d rotatonal walk (fg. 7). I fwd = I d =I rot = (dc-)t (5) An mportant part of omn-drectonal walkng the ablty to tranton between two gat. We have found that changng phang of the leg can be olved by ung a trantonal tance, namely a tance at whch the body momentarly come to ret at a reet poton. We have choen for thee purpoe an uprght tance that poton the feet at each of the foot placement curve center pont. Although th reult n ome mall hft n body poton, the effect generally neglgble. Genetc Modellng of Wheeled Moton For the creaton of wheel-lke leg moton we ued x parameter for front and x parameter for rear leg. Thee twelve parameter determne the patal apect of a robot trajectory. The th parameter bound to temporary apect and determne the peed of paw movement and the th repreent duty factor. Duty factor can take value between ¾ and ½. x poture parameter ued for degnng the gat trajectory are preented n the table. Thee poture parameter are graphcally hown n the fgure. If poture parameter for the front and rear leg are not dentcal, the top plane of the body wll make the angle wth the ground rather than horzontally. In total, there are real-valued parameter are ued to determne a gat for the locomoton module. Table lt ome of thee parameter, whch are alo the gene for ndvdual evolved by the evolutonary algorthm. Thee parameter pecfy the poton and orentaton of the body, the wng path and the wngng rate of the robot leg, the ampltude of ocllaton of the body' locaton and orentaton. Paramet er Max value (mm) Mn value (mm) Paw moton length m 60 0 Paw moton wdth n 50 0 X coordnate of COR Xο 70-0 Y coordnate of COR Yο 0 50 Z coordnate of COR Zο 0-0 Paw rotaton angle α Paw movng peed 600m/tep 00m/tep Duty factor dc ¾ ½ Tab. Control Parameter for Gat Generaton electon determne whch ndvdual are choen for matng (recombnaton) and how many offprng each elected ndvdual produce. In our degn we ued two electon technque (operator): roulette-wheel electon, alo called tochatc amplng wth replacement and electon of the fttet technque. Recombnaton produce new ndvdual n combnng the nformaton contaned n the parent (parent - matng populaton). After recombnaton the new ndvdual of a populaton are created. We ued four recombnaton technque n parallel (Guaranteed-Unform-Croover, Guaranteed-Average, Guaranteed-Bg-Creep, Guaranteed- Lttle-Creep). Mutaton after recombnaton every offprng undergoe mutaton. Offprng varable are mutated by mall perturbaton (ze of the mutaton tep), wth low probablty. The probablty of mutatng a varable et to be nverely proportonal to the number of varable (dmenon). The more dmenon one ndvdual ha a maller are the mutaton probablty.

4 Evoluton Mechanm After generatng entre populaton the controller pck up the next member of the populaton and tart evaluaton proce by pang the gene of the elected member to the robot. On the ba of thoe gene the robot wll generate all t gat. Before executng the gat determned by populaton member gene naphot hould be taken from the CCD camera mounted m above the ptch. After each tep (rotatonal or trantonal) the robot notfe controller about completon of tep movement and about the type of movement that ha been undertaken (rotatonal or tranlator wth approprate angle). The controller take another naphot of the feld and analye the change of robot poton durng the executon of the gat. After that proce ha been repeated everal tme accordng to extent and tablty of robot movement a ftne value produced. ntuton underlyng the adaptaton mechanm that t hould alter the probablty of applyng an operator n proporton to the oberved performance of the ndvdual created by that operator n the coure of a run. The average ftne and the bet member of the current operator are then compared to average ftne and bet member of all the other operator combnaton. Accordng to th comparon weght are modfed accordng to equaton (6) and (7). ' j W = W j + ( Fj r m n j F mn * r ) + ( M r ' ' Wj Wj ( mn = Wmn m n j * r W ) j r m n j M * r W,j Weght of -th electon and j-th recombnaton operator W,j weght of -th electon and j-th recombnaton operator from the prevou generaton F j Average ftne of -th electon and j-th recombnaton operator number of electon operator () r number of recombnaton operator () M,j - ftne of the bet member produced wth -th electon and j-th recombnaton operator mn ) (6) (7) 5 Mult-Objectve Optmzaton Fg. 8 Evaluaton proce of generaton member Indvdual are elected accordng to ther ftne for the producton of offprng. Parent are recombned to produce offprng. All offprng wll be mutated wth a certan probablty. The ftne of the offprng then computed. The offprng are nerted nto the populaton replacng the parent, producng a new generaton. Th cycle performed untl the optmaton crtera are reached. Fg. 9 Evolutonary algorthm flow chart A hybrd approach that adapt the probablty of genetc operator decrbed above adopted n reproducton baed on the performance of the operator' offprng. The frt There are two prmary optmaton crtera for gat generaton we have been concerned wth. Our gat hould be fat and table at the ame tme. Thu, the complexty of the problem ncreae a the number of objectve ncreae becaue the objectve condered are often contradctory to one another (for example peed and tablty). uch complex optmzaton problem have a lot of feable oluton. However, only a few oluton among them are derable. Mult-objectve evolutonary algorthm uually try to fnd all the non-domnated oluton of an optmzaton problem wth multple objectve. Let u conder the followng mult-objectve optmzaton problem wth objectve: Maxmze (f (x), f (x), f (x), f (x)) where x a vector of gene we ue to create gat and f (x), f (x), f (x), f (x) are ftne functon to be maxmzed (forwardbackward walk, de walk, rotaton walk and tablty). eparaton of thee walk type reaonable becaue of dfferent phang ued n t mplementaton. If a feable oluton not domnated by any other feable oluton of the mult-objectve optmzaton problem, that oluton ad to be a non-domnated oluton. When the followng nequalte hold between two oluton x and y, t ad that the oluton x domnated by the oluton y: Example of non-domnated oluton for two-objectve optmzaton problem hown n Fg. 0. Mult-objectve optmzaton problem uually have everal domnated oluton (flled crcle). After entre populaton ha been teted and partal ftne functon evaluated we pre-elect only non-

5 domnated oluton and dcard domnated-member. Evoluton operator for electon, recombnaton and mutaton are then appled only on non-domnated member. produce a gat that run for approxmately 5 econd. Tme neceary to fnh one evaluaton vare becaue t neceary the robot fnh at leat everal tep before evaluaton take place. If the robot fall over, t fully capable of gettng up and contnung t executon. After executng one member evaluaton take place and the peed and tablty parameter are produced whch qualtatvely determne ftne value. The overhead camera record offet from the tartng poton each tme the robot complete a tep and notfe about t movement PC controller. On the other hand robot upplyng to the controller nformaton about type of t movement. 000 Fg. 0 Non-domnated oluton (cloed crcle) The contructon of an approprate ftne functon very mportant for the correct work of the GA. Th functon repreent the problem envronment, that, t decde how well the trng olve the problem. In order to tranform the value of partal ftne functon to the ngle ftne value of a populaton member we combne weghted value of partal ftne functon a hown n the followng equaton: f(x) = w f (x) + w f (x) + w f (x) + w f (x) (8) where f(x) the ftne functon of x, and w,w,w,w, are non-negatve weght for the partal fatnee. Thee weght atfy the followng relaton: w >= 0 for I=,,,, w +w +w +w = (9) Weghtng for the forward-backward moton, de-walk and rotaton moton are proportonal to the frequency of t uage durng on-lne tranng, whle weghtng w for the tablty ftne ha contant value. To meaure tablty durng populaton member run we ued readng from gyro-enor. At the end of run, the varance of gyro readng calculated from equaton (0) and () below. The mnmaton of the varance and maxmaton of the acheved peed are targeted. EX = G = [ X, EY = Y, EZ = ( X EX ) + ( Y EY ) + ( Z EZ ) ] Z (0) () where the number of ample from the gyro. AIBO robot gyro enor return double value rangng from -9 to 9. Durng executon we ample readng from the gyro enor. Our am to get the gat wth mnmal ocllaton but t alo hould be a fat gat. tablty mportant becaue the robot hould track the ball well even when he movng. The number of ample taken from the gyro durng the robot run wth one gat vare between 0 and 0. 6 Expermental Reult The evaluaton take place nde of AIBO robot football ptch. In th experment we ued Eex Rover football playng code durng evaluaton. Each generaton member Bet Generato n Average 5 50 Trend Lne Fg. The reult of ftne durng gat generaton baed on GA When we ued a populaton ze of 0 and run t for 0 generaton, mot of the populaton member ddn t perform very well at the begnnng. ome of them ddn t even walk n the traght lne or they even walked backward. ome of them were alo caung the robot to fall. By the end of the experment, the performance of the member gnfcantly mproved. At frt tablty meaure tend to have contant growth but t ha negatve affect on the peed ncreae. tll at that tme peed ha more than doubled t value from the tart of the experment. In the th generaton tablty reache t maxmum howng no ncreae and even tendng to fall toward the end of the experment. On the other hand mall decreae n tablty value ha potve effect on the ncreae of peed of the robot. It nteretng to ee change n the operator probablte durng th experment. All operator probablte have the ame value at the begnnng, but durng the experment executon more ucceful one n mprovng average ftne tend to gan greater chance of beng ued. From thee reult (Fg ) we can ee that the bet reult are obtaned from fttet-electon and Average recombnaton operator that became domnant epecally toward the end of the experment. We could have expect that becaue averagng gene can t produce very dvere populaton and populaton mlar to parent ued to have good average ftne and ueful for earch of the local maxm but t not very ueful for explorng whole earch pace. Anyway, prmary role of mutaton to mantan a dvere populaton n order to prevent premature convergence and acheve a well-dtrbuted, wde pread trade-off front.

6 Operator Probablty 5 0 Generaton W W W W W W W W Roulette-Wheel el. of the fttet Index value j mall Bg Unform Average Creep Creep Croover Index value Fg. Operator Probablte W,j preent electon probablty of the -th electon and j-th recombnaton operator The prmary goal of creatng varable operator probablte wa to put forward thoe operator that produce better average ftne. To determne the extent we have done t we performed another et of experment. From the prevou experment we took the eed at 0 th, 0 th and 0 th generaton and then appled the ame experment but now only ung one par of operator at a tme for the perod of one generaton of 0 member. After determnng ftne value f,j for every par of operator we recalculated derable weght of thoe operator WD,j. By derable I mean that thoe operator that have greater ftne value have alo proportonally greater weght but tll mantanng um of all weght to be. The reult of the experment are hown n the table. eed from generaton 0 Exper W W W W W W W W -ment t nd eed from generaton 0 t nd eed from generaton 0 t nd Tab. eed from dfferent generaton Two mportant concluon can be derved from th experment. ome recombnaton operator contrbute more than other to the average ftne of a populaton and contrbuton of operator probablte change over tme. Better reult n term of ncreang the peed of evoluton can be accomplhed by creatng varable recombnaton and mutaton operator probablte. 7 Concluon Th paper preent a hybrd ytem for ony robot to learn good walkng behavor wth lttle or no nteracton wth the degner. Once the learnng method put nto place, the module can learn through t nteracton wth the real world. The mutatng and combnaton behavor of the Genetc Algorthm allow the proce to develop to a ueful Behavour over tme. The reultng gat from th tranng proved to be a better oluton than the nonnterference tranng for movement over all type of urface, pontng to a local optma beng dcovered n the non-envronmental nterference tuaton. The behavour of thee algorthm tochatc o they may potentally preent dfferent oluton n dfferent run of the ame algorthm. The mechanm decrbed here ha everal feature that hould be noted. It allow rapd parameteraton of operator probablte acro the range of potental genetc algorthm and operator et. It talored to a teady tate reproducton cheme. It would not be lterally applcable to problem wth noy evaluaton functon. Reference [] R. Beer, J. Chel, and L. terlng. Evoluton of platc neurocontroller for tuated agent. An artfcal nect, Amercan centt, 79, -5, 989. [] H. Crue, C. Bartlng, J. Dean, M. Drefert, T. Kndermann, and J. chmtz. Walkng: a complex behavor controlled by mple network. Adaptve Behavor,, 85-8, 995. [] D. Floreano and F. Mondada., Genetc evoluton of a neural-network drven robot. Proc. of the rd Internatonal Conference on mulaton of Adaptve Behavor, MIT Pre, -0, 99. []. Nolf. Evolvng non-trval behavor on real robot: A garbage collectng robot. Robotc and Autonomou ytem,, 87-98, 997. [5] G. Taga, Y. Yamaguch, and H. hmzu. elforganzed control of bpedal locomoton by neural ocllator n unpredctable envronment. Bologcal Cybernetc, 65, 7-59, 99. [6] M. Maza, J. Fontane, M. Armada, P. Gonzalez, Wheel+Leg- A New oluton For Tracton Enhancement Wthout Addtve ol Compacton, IEEE Robotc and Automaton Magazne, June 998, pp. 6-. [7] D. Golubovc and H. Hu, An nteractve oftware envronment for gat generaton and control degn of ony legged robot, Proceedng of Internatonal ympoum on RoboCup, Fukuoka, Japan, June 00. [8] M. Hardt and O. von tryk: Toward Optmal Hybrd Control oluton for Gat Pattern of a Quadruped. CLAWAR: Internatonal Conference on Clmbng and Walkng Robot (000) 85 9 [9] K. DeJong, An Analy of the Behavor of a Cla of Genetc Adaptve ytem, PhD Dertaton, Unverty of Mchgan, 975. [0] J. Grefentette, Optmzaton of Control Parameter for Genetc Algorthm, IEEE Tranacton on ytem, Man, and Cybernetc, Vol. MC-6, No., January/Febuary, 986.