Unmanned Aerial Vehicle (UAV) is becoming an integral

Transcription

1 Hybrd Partcle Swarm Optmzaton and Genetc Algorthm for Mult-UAV Formaton Reconfguraton Habn Duan, Qnan Luo, and Guanun Ma State Key Laboratory of Vrtual Realty Technology and Systems, Behang Unversty (BUAA), Beng, PR CHIA Yuhu Sh X an Jaotong-Lverpool Unversty, Suzhou, PR CHIA I. Introducton Unmanned Aeral Vehcle (UAV) s becomng an ntegral part of future mltary forces and wll be used for complex tasks ncludng survellance, reconnassance, precson strke and aeral refuelng mssons n the presence of dsturbances, falures, and complcated battlefeld subected to uncertantes and varatons. Therefore, more attenton s now pad to varous control problems assocated wth mult-uav movng n formaton [] []. The benefts of formaton flght (as shown n Fg. ) nclude fuel savngs at certan close formaton postons, tanker formaton operatons where flghts of UAVs are ferred by a sngle tanker, and msson success n terms of redundancy and Dgtal Obect Identfer.9/MCI..677 Date of publcaton: July 6 IEEE Computatonal ntellgence magazne August 6-6x//$. eee

2 Abstract Gven the ntal state of an Unmanned Aeral Vehcle (UAV) system and the relatve state of the system, the contnuous nputs of each flght unt are pecewse lnear by a Control Parameterzaton and Tme Dscretzaton (CPTD) method. The approxmaton pecewse lnearzaton control nputs are used to substtute for the contnuous nputs. In ths way, the mult-uav formaton reconfguraton problem can be formulated as an optmal control problem wth dynamcal and algebrac constrants. Wth strct constrants and mutual nterference, the mult-uav formaton reconfguraton n -D space s a complcated problem. The recent boom of bo-nspred algorthms has attracted many researchers to the feld of applyng such ntellgent approaches to complcated optmzaton problems n mult-uavs. In ths paper, a Hybrd Partcle Swarm Optmzaton and Genetc Algorthm (HPSOGA) s proposed to solve the mult-uav formaton reconfguraton problem, whch s modeled as a parameter optmzaton problem. Ths new approach combnes the advantages of Partcle Swarm Optmzaton (PSO) and Genetc Algorthm (GA), whch can fnd the tme-optmal solutons smultaneously. The proposed HPSOGA wll also be compared wth basc PSO algorthm and the seres of expermental results wll show that our HPSOGA outperforms PSO n solvng mult-uav formaton reconfguraton problem under complcated envronments. fotosearch battle damage assessment. Therefore, formaton control s becomng more and more mportant. There are three man approaches to formaton control, namely Leader-Wngman, Vrtual Leader, and Behavoral Structures, whch have been studed thoroughly [] []. In the Leader-Wngman structure, one of the UAVs n the formaton s desgnated as the leader, wth the rest of the UAVs (Wngmen) treated as followers. The basc dea s that the followers track the poston and orentaton of the leader. Because of ts smplcty, the Leader-Wngman structure s wdely used n control and management of mult-vehcle formatons [] [9]. The common weaknesses as reported n [] [9] are that the rear UAV usually exhbts a poorer response than ts reference due to error propagatons and the leader s a sngle pont of falure for the formaton. In the Vrtual Leader structure, the entre formaton s treated as a sngle structure. Each UAV receves the same nformaton, whch s the traectory of the Vrtual Leader. It has been appled to formatons of spacecraft n []. The strength of the Vrtual Leader structure s that t s easy to prescrbe the formaton behavor. However, ts dsadvantage s that there s no explct feedback to the formaton. A dfferent strategy s represented by a Behavoral approach. The basc dea of ths approach s to prescrbe several behavors for each arcraft and to make the control acton of each arcraft a weghted average of the control for each behavor. The behavors may be obstacle avodance, collson avodance, target seekng, and formaton keepng. It was frst ntroduced by Anderson and Robbns [] before t was further exploted by Gulett [] wth the ntroducton of an magnary pont n the formaton called the Formaton Geometry Center (FGC). Asada, et al [] successfully used a behavoral-based control to dsplay soccer-playng robots. Snce the sensors used for data acquston ntroduce nose nto the system, the handlng of sensor data needs to be mproved. The applcaton of behavoral approach to arcraft formaton flght s descrbed n [], where the control strateges are nspred by nstnctve behavors of brds, fshes, nsects, and herds. Balch and Arkn [] presented a behavoral-based approach to robot formaton keepng, where control strateges are formed by averagng several competng behavors. In [], obstacle avodance whle keepng formaton s acheved usng basc behavors, such as move-to-goal, avod-obstacle, mantan-relatve-dstance, mantan-relatve-angle, and stop move to the target destnaton. However, n [], [], [], the common shortcomngs are that the nformaton needed are too plentful and the characterstcs of the formaton (lke stablty) cannot generally be guaranteed. Many researchers have nvestgated the formaton control problems, but very few lteratures focus on the reconfguraton ssue of UAV formaton [6]. Reconfgurable control may be needed n cases of falure n one or more communcaton channels [6], sensor/actuator falures [7], flght path constrants or even the total loss of the arcraft wthout mparng ts msson. Ths work manly focuses on formaton reconfguraton problem, whch was frst addressed by Wang and Hadaegh [8]. The concept of formaton reconfguraton nvolves determnng arcraft separaton dstance, poston and orentaton, dentfyng the process that optmally transforms an ntal formaton confguraton nto a fnal confguraton and dentfyng cooperatve autonomous control (CAC) of ndvdual arcrafts to acheve a desred fnal confguraton. Determnng the transformaton process ncorporates three man technologes: constrant satsfacton, qualty determnaton, and plannng. Formaton reconfguraton can be classfed nto two basc types. For type, each arcraft s requred to occupy a specfed poston n the desred reconfgured formaton, whle for type, a specfed August IEEE Computatonal ntellgence magazne 7

3 Reconfguraton S Form Dsform Reon Splt Reconfguraton S S S Reconfguraton S Fgure Mult-UAV formaton. poston n the desred reconfgured formaton may be occuped by any arcraft of a partcular type. Several approaches have been appled to the formaton reconfguraton problem. By formulatng the formaton reconfguraton process nto a sequence of basc maneuvers, Wang and Hadaegh [9] obtaned smple solutons for formaton reconfguraton. Ferro and Das [] consdered the sngulartes and collson constrants to dynamcally reconfgure the team of autonomous robots. Wang and Zheng [] presented a Herarchcal Evolutonary Traectory Planner (HETP), whch has two levels that perform global plannng and multple optmal or near optmal traectory desgns respectvely for spacecraft formaton reconfguraton n -D space. Sauter and Palmer [] developed a sem-analytc approach and appled t to satelltes for a rapd onboard, fuel-mnmzed, and collson-free path generaton, whch can sgnfcantly ncrease the responsveness of the formaton to reconfguraton events. Ma et al. [] desgned a general formulaton for tme optmal traectory plannng of satelltes formaton reconfguraton usng pseudospetral method, and the optmzaton onlnear Programmng (LP) problem was solved by searchng the state and control vectors to mnmze an obectve functon. -D potental feld method was used n [] to solve formaton flght and formaton reconfguraton of mult-uavs, ncludng obstacle and collson avodance. By followng the nspratonal prevous works, lots of evolutonary computaton methods have been proposed, developed and studed for scentfc research and engneerng applcatons []. Gueret et al. [6] appled soft computng methods to two of the Semantc Web reasonng tasks; an evolutonary approach to queryng, and a swarm algorthm for entalment. Muhlesen and Dentler [7] proposed a novel concept for reasonng wthn a fully dstrbuted and self-organzed storage system based on the collectve behavor of swarm ndvduals. Meng et al. [8] presented a herarchcal mechanochemcal model for self-reconfguraton of modular robots n changng envronments. Le et al. [9] also presented a theoretc model of symbotc evoluton for the desgn of water clusters potental model. The formaton reconfguraton problem can be formulated as an optmal control problem wth dynamcal and algebrac constrants. For ths reason, artfcal ntellgence algorthms and/or other optmzaton methods can be utlzed to fnd the optmal soluton [], []. Furakawa et al. [] presented a method, where the control strateges are based on the Control Parameterzaton and Tme Dscretzaton (CPTD) method, to solve the tme-optmal control of the relatve formaton of mult-robotc vehcles. However, n ther method, tme s a fxed value and t s not always possble to obtan the optmal soluton at the gven value. Xong, et al. [] proposed an mproved Genetc Algorthm (GA), whch ncorporates CPTD, for mult-fghter formaton reconfguraton optmzaton. In [], tme s an optmzaton parameter but ths method can only be used n -D envronments, whch lmts ts applcaton range. The mproved GA can be used to transform the problem of tme-optmal control for formaton reconfguraton nto dscrete optmzaton problem wth a free termnal state constrant by CPTD, and the expermental results show that t s sutable for obtanng the global optma of tme-optmal control of formaton reconfguraton n two dmensons. In ths paper, formaton reconfguraton n -D space s modeled as a parameter optmzaton problem and a Hybrd Partcle Swarm Optmzaton and Genetc Algorthm (HPSOGA) s proposed. There are two crtcal ssues n the 8 IEEE Computatonal ntellgence magazne August

4 basc GA, one s ts premature convergence, and the other s ts weak local searchng ablty. Furthermore, GA also suffers from a slow convergence speed. Partcle Swarm Optmzaton (PSO) s an effcent optmzaton algorthm for solvng complcated contnuous problems. PSO s smlar to the contnuous GA n that t begns wth a random populaton matrx. Unlke the GA, PSO has no evoluton operators such as crossover and mutaton. One of the most obvous advantages of PSO over GA s ts algorthmc smplcty as t uses a few parameters and s easy to mplement. PSO can often locate nearly optmal solutons wth a fast convergence speed, but usually fals to adust ts velocty step sze for fne tunng n the search space, whch often leads to premature convergence. Combnng the advantages of PSO and GA, our hybrd approach can fnd tme-optmal solutons smultaneously. Seres of comparatve results also show that our proposed HPSOGA outperforms PSO. The rest of ths paper s organzed as follows. Secton II ntroduces the basc prncples of GA and PSO. Secton III gves a descrpton of tme-optmal control problems for mult-uav under the free termnal state constrant. Secton IV ntroduces the applcaton of HPSOGA to formaton reconfguraton. Secton V descrbes the HPSOGA mplementaton to solve the optmzaton problem. The expermental results are gven n Secton VI, followed by concludng remarks n Secton VII. II. The Standard GA and Standard PSO A. Standard Genetc Algorthm GA was frst ntroduced by Holland n the early 97s []. Generally, GA comprses three dfferent phases n the global searchng process: Phase : creatng an ntal populaton. Phase : evaluatng a ftness functon. Phase : producng a new populaton. A genetc search starts wth a randomly generated ntal populaton, wthn whch each ndvdual s evaluated by means of a ftness functon. Indvduals n ths and subsequent generatons are duplcated or elmnated accordng to ther ftness values. Indvduals are further manpulated by applyng GA operators. There are usually three GA operators n a typcal genetc algorthm [8]. The frst s the producton operator (eltsm) whch makes one or more copes of any ndvdual, wth a hgh probablty that possesses a hgh ftness value, conversely the ndvdual wth a low ftness value s elmnated wth a hgh probablty from the soluton pool. The second operator s the crossover operator. Ths operator selects two ndvduals from the populaton n the current generaton and a crossover pont (takng the onepont crossover for example) and carres out a swappng operaton on the elements to the rght hand sde of the crossover pont of both ndvduals. The thrd operator s the mutaton operator. Ths operator acts as a background operator and s used to explore some of the nvested ponts n the search space. Snce frequent applcaton of ths operator would lead to a completely random search, a very low probablty s usually assgned to ts operaton. B. Standard PSO The PSO was formulated n terms of socal and cogntve behavor by Kennedy and Eberhart n 99 [6, 7], and has found wde applcatons n engneerng. The PSO algorthm smulates socal behavor among brd ndvduals (partcles) flyng through a mult-dmensonal search space, each partcle representng a pont. The partcles assess ther postons by a ftness functon and partcles n a local neghborhood share memores of ther best poston, whle usng those memores to update ther veloctes and postons. Partcle updates n basc PSO are accomplshed accordng to () and () [8]. Equaton () calculates a new velocty for each partcle based on ts prevous velocty ( vd ), the partcle s locaton at whch the best ftness has been acheved ( pd ) so far, and the best partcle among ts neghbors ( pgd ) at whch the best ftness has been acheved so far. Equaton () updates each partcle s poston ( xd ) n the soluton hyperspace. The two random numbers r and r are ndependently generated and c and c are learnng factors. The use of the nerta weght w provdes mproved performance. vd = wvd + c$ r$ ( pd - xd) + c$ r$ ( pgd - xd) () xd = xd + vd. () There are three parts on the rght sde of (). The frst part s the velocty part, whch represents the nfluence of the prevous velocty of the partcle. The second part s the cognton part, whch represents the prvate thnkng of the partcle. The thrd part s the socal part, whch represents the collaboraton of the partcles. III. Tme-Optmal Control for Mult-Uav A. Equaton of Moton Assume the number of UAV n formaton s. The termnal tme s t = T. T s not a gven value but a parameter that should be optmzed. The th UAV s control nputs (ncludng thrust, load factor, bank angle) are represented as r u = " u() t 6t! [, T],!, 6! ", g,,. The formaton control nput vector s U = ^u, g, uh, whle the contnuous control nput vector of the formaton can be descrbed as U = ^u, g, uh = " U() t 6 t! [, T ],. The th T 6 UAV s state s X = [ v, c,, x, y, z]!, 6!", g,,, where ( x, y, z ) denotes ts coordnates and v, c, denote ts arspeed, flght path angle and headng angle respectvely. Therefore, the formaton system state can be defned as T T T 6 ) X = ^x, g, xh!. Consder a non-lnear system n the standard form []: Xo () t = f(, t X(), t Ut ( )). () August IEEE Computatonal ntellgence magazne 9

5 Gven a set of contnuous control nputs U and the ntal state X() = X, the state of the system at any tme t! (, T] can be determned unquely n the form: t X() t = X() + # - f(, x X(), x U( x)) dx. () Ths means that gven the ntal state X(), the state X(t) can be specfed only by the control nputs U n the form X( tu ). B. Obectve Functon and Constrants It s well-known that the canoncal form of the obectve functon can be expressed as []: T J( U) = U( X( TeU)) + # L(, t X( teu), U( t)) dt. () The problem may also be subect to a varety of other constrants, generally n the form: # g( U) = U( X( xeu)) + x L(, t X( teu), U( t)) dt # 6! ", g, M,. For a sngle system, the optmal control problem can be formulated as fndng the contnuous control nputs U and termnal tme T that mnmze the obectve functon J(U ): (6) mng mn J( U) (7) u, T u, T mn J ( UT, ). (8) The functon U and tme T are normally constraned by the followng equaton. Umn # U() t # Umax, 6t! [, T), T. (9) Defnng the mth UAV as the formaton center, the free termnal constrant s gven by: / = g( U, D t) = {[( x( T) -xm( T)) -x m ] [( y( T) ym( T)) y m ] [( z( T) zm( T)) z m ] } =, () where m! ", g,,, [ x m, y m, zm ] T represents the desred relatve coordnates of the th UAV wth respect to the mth UAV. The dstance between any two UAVs and s defned to be:, d ( x( t), x()) t = ^x() t - x() t h + ^y() t - y() t h. () + ^z() t -z() t h, In order to avod collson, d ( x( t), x ()) t must be greater than the safety collson dstance Dsafe., d ( x( t), x()) t $ Dsafe, 6t! [, T], 6!,!", g,,. () In order for real-tme communcaton between the UAVs to update one another on the combat stuaton of the, formaton, d ( x( t), x ()) t must be smaller than the communcaton dstance., d ((), r t m()) t $ Dcomm 6t! [, T] 6!,! ", g,,. () IV. Hpsoga Based Formaton Reconfguraton Tme-Optmal Controller PSO and GA are global optmzaton algorthms and are sutable for solvng optmzaton problems wth lnear or non-lnear obectve functons; Therefore, they are sutable for solvng non-lnear formaton reconfguraton problem. However, the control nputs of each flght unt are contnuous and the HPSOGA cannot solve the contnuous control nput problem. In order to solve ths problem, the control nputs of each flght unt are pecewse lnearzed, and the approxmaton pecewse lnearzaton control nputs are used to substtute the contnuous nputs, then HPSOGA s used to fnd the global optmal soluton. Based on the above deas, ths paper adopts the CPTD method, obtanng the approxmate obectve functon and constrants condton, smplfyng the problem n descrpton and handlng, and then usng HPSOGA to fnd the approxmate soluton Ut (; tnp, X) untl satsfyng the constrants of Eq. (), (), (8), (9) and (). A. Formaton Reconfguraton Tme-Optmal Control Dscrete Based on CPTD Method The contnuous control nputs u are approxmated by a pecewse functon wth a set of statc parameters (n practce these statc parameters are constants). The termnal tme T s frst parttoned nto n p tme ntervals. Parttonng s conducted to ntroduce a pecewse functon wth n p constants that substtute the contnuous control nputs. The termnal tme T s formulated as a functon of tme nterval D t p, whch s used for numercal ntegraton. The statc control parameter s set and the tme ntervals are found by mnmzng the obectve functon wth a standard non-lnear parametrc optmzaton method. The proposed method takes three steps to derve ths approxmate soluton of the problem. The followng subsectons descrbe these steps. ) The dvson of the termnal tme T: The termnal tme T s parttoned nto n p! {,, g} ntervals, each Dt p! +, so T = npdt p. () At each tme nterval D t p, accordng to the correspondng control nputs, equaton () does numercal ntegraton. ) The pecewse lnearzaton of control nputs: For the n p ntervals, defne r# np constants for the th UAV as r X = " v! 6!", g, np,,, 6! ", g,,. Then, IEEE Computatonal ntellgence magazne August

6 each of the contnuous control nputs for the th UAV can be approxmated by a pecewse functon wth constant as follows []: n p ut (; t np, X ) = / v () t, u() t, () where () t s gven by = ( -) Dtp # t # Dt p () t = ' (6) otherwse. Defne the set of all pecewse constants for all UAVs as X = " X, g, X,. The set of approxmated control nputs for all the UAVs can be wrtten as Ut (; tnp, X ) = " ut(; t np, X), g, ut(; tnp, X),. Fndng Ut (; tnp, X) therefore results n fndng the parameter set X. The most mportant thng for ths approxmaton n practcal mplementatons s an approprate choce for n p. Increasng n p results n an exponentally ncrease n computaton tme, whle reducng n p results n loss of accuracy. ) Approxmaton of control nputs: The approxmaton Ut (; tnp, X) = " ut(; tnp, X), g, ut(; tnp, X), can be derved from X and D t p. In fact, fndng Ut (; tnp, X) and T s equvalent to fndng X and D t p ntroduces an approxmate obectve functon and constrant functon J. As a result, the dynamc optmzaton problem Xo () t, f(, t X(), t Ut ( t; n p, X)) can be transformed nto the statc optmzaton problem: Subect to bounds J, mn( npdt p). (7) X, Dt p ( umn) # v # ( umax) 6! ", g,,, 6! ", g, np,, Dt p. (8) And the free termnal constrant: t g( X, Dt) = / {[( x( T) -xm( T)) -x m ] = [( y( T) ym( T)) y m ] [( z( T) zm( T)) z m ] } =. (9) The state of the system can be approprately wrtten as follows: Xt o () = ftxt (, (), Utn t ( ; p, W)). () umercally, the formaton reconfguraton problem s formulated as a parameter optmzaton problem wth a nonlnear obectve functon and constrants. It can now be solved wth the proposed HPSOGA. However, the soluton found by HPSOGA wll be near-optmal due to the CPTD method. B. Tme-Optmal Control of Formaton Reconfguraton Based on HPSOGA The constructon of a partcle s poston s as follows: X = " X, g, X, combnes wth D t p as the partcle s poston. Thus, the poston of each partcle can be expressed as P = [ X, X, g, X, Dtp]. Control parameter X s a constant v v g vn p array, that s X = > h h h H, 6!",, g,, 6! vr vr g vn r ",, gn p,, 6k! ",, gr,. v k s the kth component of ut (; t n p, X) at the th tme nterval. Because each column of X represents the control parameter of the th UAV at a partcular tme nterval, we expand X by column and combne t wth D t p, eventually straghtenng t nto a floatng pont code seres of length # np# r+. Fnally, the partcle s poston can be expressed as: X [((, = v v, g, vr), g,( vnp, vnpg, vnpr)), g, (( v, v, g, vr), g,( vn, vn g, vn r)), Dt p]. p p p p () In HPSOGA, the poston and velocty are randomly ntalzed. The obectve functon can be calculated as follows: Consderng the tme-optmal control constrants, the extended obectve functon s defned as: J extend ) = mn{( npdtp) + v tg ( X, Dt) X, Dt p - / / = =+, + [ v max(, Dsafe -d ( x(), t x( t))) ', + v max(, d ( x( t), x()) t -Dcomm)]}, () where v and v ' are the safety dstance punshment coeffcent and communcaton dstance punshment coeffcent, respectvely. v ) s the punshment coeffcent of the termnal constrant. As long as v, v ', and v ) are large enough (must be a postve number), the prmtve obectve functon expresson (7) and the constrant condtons () () (9) wll be equvalent to (). The ftness functon n GA s f = Jextend. Therefore, the formaton reconfguraton s classfed as a constraned parametrc optmzaton problem wth non-lnear obectve functon and constrant functons. Ths can be solved wth a standard non-lnear programmng method HPSOGA, though the global optmalty of the soluton depends upon the convexty of the obectve functon and constrant functons. V. Hybrd Algorthm Descrpton Based on the above descrpton, HPSOGA can solve formaton reconfguraton problem. The algorthm can be dvded nto two stages, the PSO stage and the GA stage. The solutons can be found by the followng steps: Step : Intalze M partcles randomly, the max teraton tme cmax, and the parameters n HPSOGA. The crossover probablty and mutaton probablty are.9 and. respectvely. August IEEE Computatonal ntellgence magazne

7 Start Randomly Intalze the Populaton wth M Partcles Compute the Ftness of Each Partcle and Record It, or Update the Poston and Ftness Fnd the Global Best Soluton, Whch Has the Mnmum Obectve Functon, and Record It Stoppng Condton Satsfed? Y Output the Optmal Soluton Populaton Is Dvded nto Two Smaller Populatons Based on the Hybrd Probablty P Evaluate Partcles Obectve Functon Values Evaluate Partcles Obectve Functon Values Update the Partcle s Velocty Select Partcles Based on Ftness Update the Partcle s Poston Perform Crossover Update Partcle s Best Soluton Perform Mutate Have All Partcles Been Updated? Y Have All Partcles Been Updated? Y Fnd the Best Soluton Fgure The detaled flow chart of HPSOGA. Step : Calculate the obectve functon values of all partcles, store the poston of the partcle wth the mnmum obectve functon value as the global best partcle. Step : Dvde the partcles nto two groups based on the hybrd probablty P, one group, wth an expected sze of M) P, uses PSO to update ther postons and the other group uses GA. Step : PSO stage. Partcles update ther veloctes and postons accordng to () and (). Two pseudo-random sequences, r + U(,) and r + U(,) are used to effect the IEEE Computatonal ntellgence magazne August

8 Table State at ntal and termnal tme. State varable UAV UAV UAV UAV UAV new P = ~ $ P+ ( - ~ ) $ P () new P = ~ $ P+ ( - ~ ) $ P, (6) Intal state XI (km) - - Intal state YI (km) Intal state ZI (km) 6 8 Relatve state XT(km) Relatve state YT(km) x PSO-GA PSO Relatve state ZT(km) stochastc algorthm nature. For all dmensons!... n, let x,, v,,pbest, be the th dmenson of the current poston, current velocty and current personal best poston of the th partcle and Gbestpso s the global best poston of the M) P partcles. F s the current personal best partcle s obectve functon value and Gpso s the global best partcle s obectve functon value. The velocty update step s: where [9] v, = wv, + c$ r$ (pbest, -x, ) + c$ r$ (Gbest pso, - x, ), () w = { = c+ c, {. { { { The updated velocty s then added to the current poston of the partcle to obtan the new poston: x, = x, + v,. () Then, we can get the new obectve functon value of x and record t as F '. If F ' s less than F, the current personal best partcle s obectve functon s F ', and the current personal best poston s the new poston. If F ' s less than Gpso, the global best partcle s obectve functon value s F ' and the global best poston s the new poston. Step : GA stage. GA has three operators, namely selecton, crossover, and mutaton, descrbed as follows: ) Selecton Operator Roulette wheel selecton strategy s wdely used n GA because t can ensure that the selecton probablty of each partcle s proportonal to ts ftness,.e. the better a partcle s ftness, the more lkely t wll be selected. ) Crossover Operator Crossover happens between two parents whch are ndependently selected from the populaton. Chldren are created by the sngle-pont crossover operaton. It can be defned as follows: J Iteraton Fgure Comparson evoluton cure of PSO and HPSOGA. z (km) x (km) - y (km) Fgure The reconfguraton traectory obtaned by HPSOGA. z (km) y (km) - x (km) Fgure The reconfguraton traectory obtaned by PSO. August IEEE Computatonal ntellgence magazne

9 where P and P are parent partcles, P new and P new are chld partcles, ~ s a random number such that ~! [,]. selected based on ts ftness. We adopt the adaptve acceleraton mutaton operator whch can be defned as follows: ) Mutaton Operator Mutaton operator can mantan partcle dversty and avod premature convergence. It s executed on a partcle whch s 6 where P ( k+ ) = P () k + b$ DP () k + t$ sp () k, (7) best DP () k = ( P ( k) -P ( k)) $ (,) sp ( k+ ) = b$ acc () k $ DP ( k) + t$ sp ( k). y (km) P () k s the th dmenson of the th partcle n the kth generaton, P () k s the best ndvdual n the kth generaton. t and best b are the learnng speed and nerta constant respectvely, and they are set as.6 and. based on tral experments. (,) s the normal random dstrbuton functon, and acc () k s defned as follows: - x (km) Fgure 6 The horzontal traectory obtaned by HPSOGA., f new ftness greater than before acc () k = ' (8), else. The poston Gbestga and obectve functon value Gga of the best partcle that the GA can fnd are stored. The best obectve functon value s the recprocal of the maxmum ftness. 6 y (km) z (km) - - x (km) Fgure 7 The horzontal traectory obtaned by PSO. Fgure 9 The horzontal traectory obtaned by PSO. z (km) d (km) Communcaton Dstance Colson Avodng Dstance Fgure 8 The horzontal traectory obtaned by HPSOGA.... Fgure The dstance of any two UAVs obtaned by HPSOGA. IEEE Computatonal ntellgence magazne August

10 Step 6: Comparng Gpso and Gga, f Gpso s less than Gga, the global best poston s Gbestpso and Gbestga s replaced by Gbestpso and Gga by Gpso. Else, the global best poston s Gbest ga, replace Gbestpso by Gbestga and Gpso by Gga. Step 7: Repeat step to step 6 untl the endng condton s met. The flow chart of our proposed HPSOGA s shown n Fg.. VI. Experments and Analyss Consderng UAVs, the flght alttude of the th UAV s T 6 denoted as = [ v, c,, x, y, z]!, 6!", g,,, where x, y, and z are the coordnates of the center of the th UAV. The UAV model s smplfed n ths formaton reconfguraton problem, and the outer loop varables such as thrust T, load factor n, and bank angle z are chosen as control nputs ( u ) to each UAV. The equatons of moton for the th UAV are as follows []: : v = g[( T-D)/ W- sn c] : c = (/)( gv ncosz-cos c) : = ( gnsnz)/( vcos c) : x = vcosc cos : y = vcosc sn (9) () () () () : c z =-v sn, () where D s the aerodynamc drag, c s the flght path angle, s the headng angle, and W s the weght of th vehcle, = g,,,. The goal of ths problem s to fnd the contnuous control nputs such that the UAVs, startng from an arbtrary ntal state T (kg) Fgure Computed optmal thrust force obtaned by PSO. d (km) Communcatons Dstance n Colson Avodng Dstance -.. Fgure The dstance of any two UAVs obtaned by PSO Fgure Computed optmal load factor obtaned by HPSOGA. T (kg) 8 6 n Fgure Computed optmal thrust force obtaned by HPSOGA. Fgure Computed optmal load factor obtaned by PSO. August IEEE Computatonal ntellgence magazne

11 PSO and GA are global optmzaton algorthms and are sutable for solvng optmzaton problems wth lnear or non-lnear obectve functons; Therefore, they are sutable for solvng non-lnear formaton reconfguraton problem. ( x ( )!,...,! x ( )), termnate wth a relatve formaton at the optmal tme T. The desred relatve coordnates for each UAV are gven wth respect to those of a UAV located n the center. The relatve coordnates of the central UAV are thus [,, ]. In our experment, =. It means that there are fve UAVs. M =, cmax = 7, P =., c = c =., the ntal value of w = 8., Pc = 9., Pm =., t =, b =.. Assume that the thrd vehcle s the center of the formaton. Dsafe = (km), Dcomm = (km). The ntal states of the UAVs are set randomly and the relatve states at tme t = T are shown n Table. After the optmal control, the UAVs can move to the desred relatve of V-shape formaton at the same alttude. Compared wth the standard PSO, the expermental results show that our proposed HPSOGA can obtan better solutons. z (deg) Fgure 6 Computed optmal headng angle obtaned by HPSOGA. z (deg) Fgure 7 Computed optmal headng angle obtaned by PSO. Fg. descrbes the relatonshp of the obectve functon and teraton count of PSO and HPSOGA. From Fg., we can conclude that the HPSOGA performs better than PSO. Fg. shows the formaton reconfguraton traectory of the soluton obtaned by HPSOGA. The numbers ndcate the flght traectores of the respectve UAVs. o s the ntal state and * s the termnal state. Fg. s the formaton reconfguraton traectory of the soluton obtaned by PSO. Fg. 6 dsplays the horzontal traectory of the soluton obtaned by HPSOGA, Fg. 7 gves the horzontal traectory of the soluton obtaned by PSO. From Fg. 6, we can see that the vehcles have successfully moved to the desred relatve V-shape formaton. However, the UAVs faled to move to the desred relatve V-shape formaton n Fg. 7. Fg. 8 shows the vertcal traectory of the soluton obtaned by HPSOGA, whle Fg. 9 gves the vertcal traectory of the soluton obtaned by PSO. From Fg. 8, t can be seen that the UAVs are at almost the same alttude (although there are errors) at the end of the traectory reconfguraton process but from Fg. 9, t s obvous that the UAVs are not at the same alttude. The dstance between any two UAVs for the soluton obtaned by HPSOGA s gven n Fg., and the correspondng results for the soluton obtaned by PSO are shown n Fg.. Fg. llustrates that the dstance between any two UAVs always satsfes the constrants () and (), whle from Fg., t can be concluded that the PSO soluton does not always satsfy the constrants () and (). Fg. shows the computed optmal thrust forces for all the UAVs for the soluton obtaned by HPSOGA, whle the computed optmal thrust forces for all the UAVs for the PSO soluton are shown n Fg.. Fg. and Fg. show the computed optmal load factors for all the UAVs of the solutons obtaned by HPSOGA and PSO, respectvely. The computed optmal headng angles for all the UAVs of the soluton obtaned by HPSOGA are shown n Fg. 6. Fg. 7 gves the computed optmal headng angles for all the UAVs of the soluton obtaned by PSO. From the smulaton results, we can conclude that the UAVs based on the HPSOGA successfully move to the desred relatve V-shape formaton n -D space, whle the PSO soluton fals. Ths result ndcates that the HPSOGA has hgher search veracty, more rapd convergence speed, and stronger ablty aganst precocty than PSO. The hybrd algorthm utlzes the advantages of both GA and PSO n solvng optmzaton problems. VII. Conclusons Mult-UAV s often requred to change ts relatve formaton from one to another n the battlefeld envronment. In addton to the termnal state constrant and control acton energy constrants, the constrants of safe dstance to avod collson and dependable dstance to guarantee normal communcaton between each par of UAVs are also consdered. On the premse 6 IEEE Computatonal ntellgence magazne August

12 of satsfyng all the above constrant requrements, dfferent optmal obectve functons ft n wth dfferent mult-uav formaton reconfguraton optmzaton problems. A method to solve the mult-uav formaton reconfguraton problem n -D space has been presented. The problem was formulated as an optmzaton problem nvolvng the mnmzaton for a specfed obectve functon wth state relatve constrants. The CPTD method has been used to solve ths problem wth a free termnal state constrant. Formaton reconfguraton problem was focused on determnng optmal control nputs for each UAV such that the group can start from the ntal state and reach ts fnal confguraton at the optmal tme whle satsfyng the set of constrants. The experments were also conducted to show that the proposed HPSOGA can successfully solve the optmal control for the mult-uav formaton reconfguraton problem n -D space. Gven a random ntal state and the target relatve state at the termnal tme, the HPSOGA can fnd the optmal soluton to meet the obectve functon requrements and varous formaton system constrants to acheve formaton reconfguraton n -D space. The HPSOGA presented n ths paper s able to, not only solve the sngle-formaton reconfguraton problem, the mnmum energy control, and the shortest tme and mnmum energy ntegrated control problems, but also solve the centralzed control of complex systems, such as mult-formaton reconfguraton, mult-vehcle coordnate problems. Acknowledgments Ths work was partally supported by atural Scence Foundaton of Chna (SFC) under grant #67, 6767, 6977, and 6978, atonal Key Basc Research Program of Chna under grant #CB, Program for ew Century Excellent Talents n Unversty of Chna under grant #CET--, Top-otch Young Talents Program of Chna, Fundamental Research Funds for the Central Unverstes of Chna, and Aeronautcal Foundaton of Chna under grant #9. References [] S. C. Felter and. E. Wu, A relatve navgaton system for formaton flght, IEEE Trans. Aerosp. Electron. Syst., vol., no., pp , July 997. [] J. R. Rehl, G. E. Collns, and J. P. Hespanha, Cooperatve search by UAV teams: A model predctve approach usng dynamc graphs, IEEE Trans. Aerosp. Electron. Syst., vol. 7, no., pp , Oct.. [] A. Franch, C. Masone, H. H. Bulthoff, and P. R. Gordano, Blateral teleoperaton of multple UAVs wth decentralzed bearng-only formaton control, n Proc. IEEE/RSJ Int. Conf. Intellgent Robots Systems, San Francsco, CA,, pp.. []. gam, S. Benawsk, I. Kroo, and J. Van, Control of multple UAVs for persstent survellance: Algorthm and flght test results, IEEE Trans. Control Syst. Technol., vol., no., pp. 6, Sept.. [] M. Pachter, J. J. D Azzo, and A. W. Proud, Tght formaton flght control, J. Gud. Control Dyn., vol., no., pp. 6, Apr.. [6] H. B. Duan, Y. P. Zhang, and S. Q. Lu, Multple UAVs/UGVs heterogeneous coordnated technque based on recedng horzon control (RHC) and velocty vector control, Sc. Chna Technol. Sc., vol., no., pp , Apr.. [7] M. Pachter, J. J. D Azzo, and J. Dargan, Automatc formaton flght control, J. Gud. Control Dyn., vol. 7, no. 6, pp. 8 8, ov. 99. [8] H. B. Duan and S. Q. Lu, onlnear dual-mode recedng horzon control for multple UAVs formaton flght based on chaotc partcle swarm optmzaton, IET Control Theory Applcat., vol., no., pp. 6 78, ov.. [9] S. S. Stankovc, M. J. Stanoevc, and D. D. Slak, Decentralzed overlappng control of a platoon of vehcles, IEEE Trans. Control Syst. Technol., vol. 8, no., pp. 86 8, Sept.. [] M. R. Anderson and A. C. Robbns, Formaton flght as a cooperatve game, n Proc. AIAA Gudance, avgaton Control Conf., Phladelpha, PA, 998, pp.. [] R. Beard and F. Hadaegh, Constellaton templates: An approach to autonomous formaton flyng, n Proc. World Automaton Congr., Anchorage, AK, 998, pp [] F. Gulett, L. Polln, and M. Innocent, Formaton flght control: A behavoral approach, n Proc. AIAA Gudance, avgaton Control Conf., Montreal, Canada,, pp. 6. [] M. Asada, E. Uchbe, and K. Hosoda, Cooperatve behavor acquston for moble robots n dynamcally changng real worlds va vson-based renforcement learnng and development, Artf. Intell., vol., no., pp. 7 9, June 999. [] T. Balch and R. C. Arkn, Behavor-based formaton control for mult robot teams, IEEE Trans. Robot Autom., vol., no. 6, pp , Dec [] R. Dougherty, V. Ochoa, Z. Randles, and C. Ktts, A behavoral control approach to formaton-keepng through an obstacle feld, n Proc. IEEE Aerospace Conf., Bg Sky, MT,, pp [6] L. Polln, F. Gulett, and M. Innocent, Robustness to communcaton falures wthn formaton flght, n Proc. Amercan Control Conf., Anchorage, AK,, pp [7] S. Thomas, B. C. Chang, and H. G. Kwatny, Reconfgurable control of arcraft wth partal elevator, n Proc. AIAA Gudance, avgaton, Control Conf. Exhbt, Austn, TX,, pp. 6. [8] P. K. C. Wang and F. Y. Hadaegh, Mnmum-fuel formaton reconfguraton of multple free-flyng spacecraft, J. Astronaut. Sc., vol. 7, no., pp. 77, 999. [9] P. K. C. Wang and F. Y. Hadaegh, Optmal formaton-reconfguraton for multple spacecraft, n Proc. AIAA Gudance, avgaton Control Conf., Boston, MA, 998, pp [] R. Ferro and A. K. Das, Hybrd control of reconfgurable robot formatons, n Proc. Amercan Control Conf., Denver, CO,, pp [] S. Y. Wang and C. W. Zheng, A herarchcal evolutonary traectory planner for spacecraft formaton reconfguraton, IEEE Trans. Aerosp. Electron. Syst., vol. 8, no., pp , Jan.. [] L. Sauter and P. Palmer, Onboard semanalytc approach to collson-free formaton reconfguraton, IEEE Trans. Aerosp. Electron. Syst., vol. 8, no., pp. 68 6, July. [] G. F. Ma, H. B. Huang, and Y. F. Zhuang, Tme optmal traectory plannng for reconfguraton of satellte formaton wth collson avodance, n Proc. 8th IEEE Int. Conf. Control Automaton, Xamen, Chna,, pp [] T. Paul, T. R. Krogstad, and J. T. Gravdahl, UAV formaton flght usng D potental feld, n Proc. 6th Medterranean Conf. Control Automaton, Aacco, France, 8, pp.. [] P. J. Werbos, Computatonal ntellgence for the smart grd-hstory, challenges, and opportuntes, IEEE Comput. Intell. Mag., vol. 6, no., pp., Aug.. [6] C. Gueret, S. Schlobach, K. Dentler, M. Schut, and G. Eben, Evolutonary and swarm computng for the semantc web, IEEE Comput. Intell. Mag., vol. 7, no., pp. 6, May. [7] H. Muhlesen and K. Dentler, Large-scale storage and reasonng for semantc data usng swarms, IEEE Comput. Intell. Mag., vol. 7, no., pp., May. [8] Y. Meng, Y. Y. Zhang, and Y. C. Jn, Autonomous self-reconfguraton of modular robots by evolvng a herarchcal mechanochemcal model, IEEE Comput. Intell. Mag., vol. 6, no., pp., Feb.. [9] M.. Le, Y. S. Ong, Y. C. Jn, and B. Sendhoff, A unfed framework for symboss of evolutonary mechansms wth applcaton to water clusters potental model desgn, IEEE Comput. Intell. Mag., vol. 7, no., pp., Feb.. [] Y. M. B. Al, An augmented partcle swarm model based b-acceleraton factor, Int. J. Intell. Comput. Cybern., vol., no., pp. 87 6, July. [] D. S. Guo and Y.. Zhang, ovel recurrent neural network for tme-varyng problems solvng, IEEE Comput. Intell. Mag., vol. 7, no., pp. 6 6, ov.. [] T. Furukawa, H. F. Durrant-Whyte, F. Bourganlt, and G. Dssanayake, Tmeoptmal coordnated control of the relatve formaton of multple vehcles, n Proc. IEEE Int. Symp. Computatonal Intellgence Robotcs Automaton, Kobe, Japan,, pp [] W. Xong, Z. J. Chen, and R. Zhou, Optmzaton for multple flght vehcles formaton reconfguraton usng hybrd genetc algorthm, n Proc. st Chnese Gudance, avgaton Control Conf., Beng, Chna, 7, pp. 6. [] J. H. Holland, Adaptaton n atural and Artfcal System. Ann Arbor, MI: Unv. of Mchgan Press, 97, pp. 6. [] J. anbo and Y. Rahmat-Sam, Hybrd real-bnary partcle swarm optmzaton (HPSO) n engneerng electromagnetcs, IEEE Trans. Antennas Propag., vol. 8, no., pp , Dec.. [6] J. Kennedy and R. Eberhart, Partcle swarm optmzaton, n Proc. 99 IEEE Int. Conf. eural etworks, Perth, Australa, 99, pp [7] R. Eberhart and J. Kennedy, A new optmzer usng partcle swarm theory, n Proc. 6th Int. Symp. Mcro-Machne Human Scence, agoya, Japan, 99, pp. 9. [8] Y. H. Sh and R. Eberhart, A modfed partcle swarm optmzer, n Proc. 998 IEEE Int. Conf. Evolutonary Computaton, Anchorage, AK, 998, pp [9] M. Clerc and J. Kennedy, The partcle swarm-exploson, stablty, and convergence n a multdmensonal complex space, IEEE Trans. Evol. Comput., vol. 6, no., pp. 8 7, Feb.. August IEEE Computatonal ntellgence magazne 7