The Pennsylvania State University. The Graduate School. Department of Electrical Engineering MULTI-OBJECTIVE OPTIMIZATION FOR UNMANNED SURVEILLANCE

Transcription

1 The Pennsylvana State Unversty The Graduate School Department of Electrcal Engneerng MULTI-OBJECTIVE OPTIMIZATION FOR UNMANNED SURVEILLANCE NETWORKS USING EVOLUTIONARY ALGORITHMS A Thess n Electrcal Engneerng by Jonathan Wllams 008 Jonathan Wllams Submtted n Partal Fulfllment of the Requrements for the Degree of Master of Scence May 008

2 The thess of Jonathan Wllams was revewed and approved* by the followng: Kwang Y. Lee Professor Emertus of Electrcal Engneerng Thess Advsor Randy L. Haupt Senor Scentst Jeffrey J. Wenschenk Research Assocate W. Kenneth Jenkns Professor of Electrcal Engneerng Head of the Department of Electrcal Engneerng *Sgnatures are on fle n the Graduate School

3 ABSTRACT The exstng lterature contans many control systems for wreless sensor and survellance networks. In the majorty of these systems, sensors are densely dstrbuted and energy conservaton s a secondary concern. It s the vew of ths work that a densely dstrbuted sensor network would have an unacceptably hgh cost and s not practcal for a large-scale underwater survellance network. Ths thess proposes a sparsely dstrbuted underwater survellance network combnng autonomous moble vehcles and fxedlocaton sensor platforms. It s the premse of ths thess that the lmtatons of the sparsely dstrbuted network can be overcome through vehcle moblty, properly placed fxed sensors, and network-level coordnaton and control. The control of ths network s dvded nto two optmzaton problems. The frst s the problem of ntal target detecton, termed drected search. Ths problem conssts of creatng vehcle search paths whch maxmze the probablty of target detecton whle mnmzng energy consumpton. A two-tered soluton approach s presented whch uses Dfferental Evoluton and Evolutonary Strateges. The second problem s that of mantanng target survellance, termed asset allocaton. The problem conssts of choosng whch vehcles wll engage n target survellance and how the survellance task wll be dvded. An optmzaton approach usng Evolutonary Strateges s presented whch maxmzes hold tme whle mnmzng energy consumpton. Smulaton results show the approach to be superor to a number of determnstc allocaton algorthms.

4 TABLE OF CONTENTS v LIST OF FIGURES...v LIST OF TABLES...x ACKNOWLEDGEMENTS...x Chapter 1 Introducton Motvaton for Research Overvew of Approach Thess Organzaton...4 Chapter Unmanned Survellance Networks Survellance Network Objectves...7. Network Composton Network Confguraton and Control Statc Networks Dynamc Networks Responsve Networks...13 Chapter 3 Control Problem Formulaton Survellance Network Objectves Survellance Network Assumptons Prncple Concept of Operaton Drecton of Contact Approach Contact Trajectory Intal Contact Detecton Dmensonalty Communcatons Autonomy Statons Survellance Network Operaton Problem Defnton and Abstracton Drected Search Asset Allocaton...33 Chapter 4 Asset Allocaton Problem Formulaton Soluton Representaton...39

5 4.1. Constrants Boundares Area of Interest Boundares Operaton Area Boundares Mnmum Intercept Tmes Combned Boundares Optmzaton Objectves Runtme Calculaton Mxed Integer Optmzaton Evolutonary Strateges Applcaton of Evolutonary Strateges to Asset Allocaton ES Soluton Format Populaton Intalzaton Recombnaton Mutaton Mutaton Operators Splt Operaton Combne Operaton Swtch Operaton Exchange Operaton Slde Operaton Shft Operaton Algn Operaton Selecton Eltsm Dversty Quotas Populaton Caps Stochastc Tournament Search Scope Global Factor Search Rate Search Termnaton Smulaton Results Smulaton Methodology Performance Effects of ES Operators Algorthm Comparson Advanced Condtons Chapter 5 Drected Search Problem Formulaton Representaton of Search Paths Optmzaton Objectves Detecton Probablty Evaluaton v

6 5..1 Gaussan Dstrbutons Uncertan Track Representaton Path Leg Detecton Probablty Total Detecton Probablty Prevous Approaches Drected Search Soluton Approach Leg-by-Leg Intalzaton Global Search Drected Search Results Drected Search Example Performance Effects of Changes n Drected Search Parameters Chapter 6 Concluson Future Work Bblography Appendx A Drected Search Appendces A.1 Samplng Footprnt Generaton A. Target PDF Intercept v

7 LIST OF FIGURES v Fgure 3-1: Example underwater survellance network deployment around coastal cty llustratng restrcted drecton of approach...19 Fgure 3-: Example underwater survellance network deployment around the entrance to a bay llustratng restrcted drecton of approach...19 Fgure 3-3: Example contact trajectory and estmaton usng pecewse-lnear model...1 Fgure 3-4: Moble node postons before uncontrolled drftng, showng ocean currents...6 Fgure 3-5: Moble node postons after uncontrolled drftng, showng exposed regon...7 Fgure 3-6: Moble nodes and staton assgnments...8 Fgure 3-7: Example drected search showng wayponts and possble moble node paths...33 Fgure 3-8: Intal condtons for example survellance response as drected by asset allocaton soluton, showng ntercept and break-off ponts...35 Fgure 3-9: Example asset allocaton response wth nodes movng to ntercept postons...36 Fgure 3-10: Example asset allocaton response showng survellance handoff...36 Fgure 3-11: Example asset allocaton response showng node trajectores as contact exts area of nterest...37 Fgure 4-1: Example asset allocaton soluton correspondng to Table Fgure 4-: Begnnng (a) and end (b) of lmted survellance wndow...44 Fgure 4-3: Example area of nterest bounded by a convex polygon...47 Fgure 4-4: Example moble node operaton areas...50 Fgure 4-5: Optmal moble node ntercept trajectory...5 Fgure 4-6: Low energy cost weghtng functon...59

8 Fgure 4-7: Runtme calculaton for ntercept segment...6 Fgure 4-8: Runtme calculaton for return segment...64 Fgure 4-9: Flowchart for asset allocaton Evolutonary Strateges algorthm...71 Fgure 4-10: Recombnaton parent PMF...78 Fgure 4-11: Example recombnaton operaton parents before recombnaton...79 Fgure 4-1: Example recombnaton operaton chld after recombnaton...80 Fgure 4-13: Local mnmum due to confguraton change ntal soluton...84 Fgure 4-14: Local mnmum due to confguraton change ntermedate solutons: node B set to actve (a), node A break-off tme altered (b)...85 Fgure 4-15: Local mnmum due to confguraton change mproved soluton...86 Fgure 4-16: Moble node travel dstance for one and two node responses...88 Fgure 4-17: Splt operaton example before operaton (a) and after (b)...90 Fgure 4-18: Combne operaton motvaton...9 Fgure 4-19: Example combne operaton before operaton (a) and after (b)...94 Fgure 4-0: Swtch operaton example before operaton (a) and after (b)...95 Fgure 4-1: Exchange operaton example before operaton (a) and after (b)...96 Fgure 4-: Less ft ntermedate soluton avoded by exchange operaton...97 Fgure 4-3: Slde operaton motvaton suboptmal handoff locaton for two nodes...98 Fgure 4-4: Slde operaton motvaton ftness score n regon of ntal soluton..99 Fgure 4-5: Slde operaton motvaton ftness score n regon of ntal soluton (small scale)...99 Fgure 4-6: Slde operaton example before operaton (a) and after (b) Fgure 4-7: Shft operaton example before operaton (a) and after (b) Fgure 4-8: Algn operaton example before operaton (a) and after (b) v

9 Fgure 4-9: Global factor search rate Fgure 4-30: Example asset allocaton problem...1 Fgure 4-31: Example asset allocaton problem response...13 Fgure 4-3: ES convergence usng dfferent parameter settngs...15 Fgure 4-33: Mean ES convergence usng dfferent parameter settngs...16 Fgure 4-34: Ftness vs. runtme scatter plot usng dfferent parameter settngs for the example problem...17 Fgure 4-35: Ftness vs. runtme scatter plot for dfferent parameter settngs (nset)...18 Fgure 4-36: Ftness hstogram for normal and fxed global factor settngs...18 Fgure 4-37: Confguraton hstogram for dfferent parameter settngs...19 Fgure 4-38: Dversty maps for lmted operatons (a) and normal settngs (b) Fgure 4-39: Vorono allocaton example...13 Fgure 4-40: k-best allocaton example Fgure 4-41: Determnstc allocaton method ftness comparson results all trals (a) and close up of nterquartle regons (b) Fgure 4-4: Advanced asset allocaton example Fgure 4-43: Asset allocaton example - ntal moble node response Fgure 4-44: Asset allocaton example moble node response after target headng change Fgure 5-1: Example drected search paths Fgure 5-: Search path leg samplng footprnt Fgure 5-3: Excluson expanson over tme Fgure 5-4: Dfferental Evoluton pseudo-code for a sngle leg of the leg-by-leg ntalzaton Fgure 5-5: Global search ntalzaton pseudo-code x

10 Fgure 5-6: Pseudo-code for mantanng constant speed and tme steps Fgure 5-7: Path adjustment example Fgure 5-8: Pseudo-code for drected search ES algorthm for global search Fgure 5-9: Node postons and target poston PDF for example DS problem (a) close up of poston PDF (b) Fgure 5-10: Target velocty PDF for example DS problem...17 Fgure 5-11: Intal and global search solutons for DS example Fgure 5-1: Target poston probablty durng drected search Fgure 5-13: Intal node postons for dfferent values of N DS Fgure 5-14: Soluton ftness and algorthm runtme for varyng N DS Fgure 5-15: Soluton total detecton probablty for varyng N DS Fgure 5-16: Soluton total path length for varyng N DS Fgure 5-17: Target poston PDF after four hours for three values of k vel Fgure 5-18: Drected search performance for varyng target velocty uncertanty (k vel ) Fgure 5-19: Ftness component scatter plot for varyng k vel wth lnes markng equal-ftness contours...18 Fgure 5-0: Drected search performance for varyng mean target speed Fgure A-1: Estmated target PDF ntercept problem...01 x

11 LIST OF TABLES x Table 4-1: Asset allocaton soluton correspondng to Fgure Table 4-: Optmal ntercept problem parameters for a sngle node...53 Table 4-3: Recombnaton of multple parent solutons to form chld...77 Table 4-4: Selecton process example...11 Table 4-5: ES parameter changes wth changng global factor Table 4-6: ES parameter values...11 Table 4-7: Example asset allocaton problem parameters...1 Table 4-8: Determnstc allocaton method comparson results superor solutons Table 4-9: Determnstc allocaton method comparson results nferor solutons..136 Table 4-10: Advanced asset allocaton example parameters Table 5-1: Drected search parameter values Table 5-: Track parameter values for example DS problem...17 Table 5-3: Intal node postons used for each value of N DS Table A-1: Samplng footprnt contour ellpse parameters...197

12 ACKNOWLEDGEMENTS x I would frst lke to thank the members of my commttee: Dr. Jeffrey Wenschenk, Dr. Kwang Y. Lee, and Dr. Randy Haupt. I would also lke to thank the Appled Research Laboratory Exploratory and Foundatonal program, whose support made ths work possble. My thanks are also extended to all my colleagues n the Intellgent Control Systems Department and the Power Systems Control Lab for ther support n ths endeavor. I would also lke to thank the members of the Genus Party for ther nsghtful advce. Fnally I would lke to express my sncere grattude to my parents and grandmother for ther patence, support, and encouragement throughout ths entre process.

13 1 Chapter 1 Introducton In recent years, ncreased nterest n homeland securty has sparked a need for mproved survellance capabltes n many stuatons and envronments. In many of these envronments, drect human survellance s not practcal due to economc, logstc, or safety concerns. Ths fact has caused an ncreased nterest n autonomous unmanned survellance networks for many organzatons. Due to the sgnfcant mprovements n autonomous vehcle capablty and wreless sensor networks n recent decades, practcal realzatons of these unmanned survellance networks are becomng a feasble possblty. One applcaton to whch unmanned sensor networks are partcularly well suted s that of undersea survellance. The nature of the undersea envronment makes drect human survellance all but mpossble. Fortunately, recent years have seen the development of numerous unmanned underwater vehcles (UUVs) whch make underwater survellance a possblty. Underwater vehcles make a practcal survellance network a realty due to certan benefts of the undersea envronment. Frst, they are not subject to the sgnfcant power requrements assocated wth most unmanned aeral vehcles (AUVs) to stay aloft. Addtonally, they are presented wth sgnfcantly fewer terran and obstacle challenges than unmanned ground vehcles (UGVs). Although underwater communcaton wth UUVs can be more dffcult than overar communcaton, on-board vehcle autonomy can be used to reduce the communcaton burden. Improvements n on-board vehcle hardware and autonomous control algorthms

14 allow the responsblty for low-level control and trackng tasks to be shfted to the vehcle, reducng communcaton overhead. Regardless of the level of on-board vehcle autonomy, a hgh-level control system s needed to coordnate the responses of multple autonomous underwater vehcles (AUVs) to the survellance task. 1.1 Motvaton for Research Ths research was motvated by the need for a hgh-level control system to coordnate the response of multple AUVs to the appearance of an unknown contact. Although much research has been done on wreless sensor networks and target trackng, untl recently lttle had been done on optmzng the responses of multple vehcles n a survellance network n terms of multple objectves. The work whch had been done on unmanned survellance networks manly focused on statc or densely dstrbuted networks n whch sensors have overlappng coverage. Due to the hgh cost of underwater vehcles, an underwater survellance network wth densely dstrbuted nodes s not realstc for wde area survellance, as such a network s nherently lmted n sze. It s the vew of ths work that the nodes n a practcal survellance network would be sparsely dstrbuted, wth the spacng between nodes greater than the range of ther sensors. Such a network would use moblty and hgh-level coordnaton to compensate for the lack of blanket coverage and provde a lower survellance cost per unt of area coverage. In order for such a network to be possble, a hgh-level control system s needed to coordnate the autonomous vehcles usng all avalable target nformaton. Rather than explctly controllng the vehcles, as n many prevous approaches, ths control system

15 3 must allocate vehcles to the survellance task and provde them wth general nstructons. The control system must balance multple objectves, energy consumpton and hold tme, when performng ths allocaton. To the knowledge of the author, no such control system currently exsts. The creaton of a control strategy for such a network s the prmary purpose of ths work. 1. Overvew of Approach The control approach presented n ths thess dvdes the overall control problem nto two ndependent sub-problems. The drected search problem consders the ssue of locatng a target usng uncertan nformaton about ts poston and velocty. The problem conssts of choosng search paths for multple AUVs such that the probablty of detectng the target s maxmzed whle consderng energy consumpton. The presented approach uses a two-tered optmzaton process usng Dfferental Evoluton and Evolutonary Strateges. The asset allocaton problem consders the ssue of optmally allocatng multple assets to a survellance task such that target survellance s mantaned whle maxmzng network lfetme. The problem conssts of choosng AUVs to perform the survellance task and dvdng ths task between all partcpatng vehcles. The presented approach uses an algorthm based on Evolutonary Strateges to perform ths allocaton.

16 1.3 Thess Organzaton 4 Ths thess begns wth a dscusson of unmanned survellance networks n Chapter. Ths dscusson starts wth an overvew of potental objectves for an unmanned survellance network. The chapter then contnues wth a defnton of fxed and moble nodes n an unmanned survellance network ncludng examples of each for multple envronments. Fnally, a summary of past approaches to unmanned survellance networks s gven. Ths conssts of network confguratons, concepts of operaton, and control approaches used by prevous authors. Chapter 3 presents the conceptual unmanned survellance network whch s studed n ths thess. In addton t presents the formulaton and abstracton of the control problem. The chapter begns wth a dscusson of the objectves of ths network. It then moves on to the assumptons regardng the network whch govern ts operaton. The effects of these assumptons are then summarzed n the operaton of the proposed network. The control problem s then defned and abstracted nto two sub-problems: mult-objectve asset allocaton and drected collaboratve search. Chapter 4 presents the formulaton and control approach for the asset allocaton problem. Ths presentaton begns wth the formulaton of the asset allocaton problem, startng wth the representaton of potental solutons. Ths also ncludes constrants and boundares on these potental solutons. An objectve functon, whch rates proposed solutons based on the chosen objectves, s also gven. The chapter then contnues wth a bref overvew of mxed nteger optmzaton and prevous approaches. The chosen approach, Evolutonary Strateges, s then dscussed. Afterwards, the applcaton of

17 5 Evolutonary Strateges to the asset allocaton problem s presented. Ths ncludes the soluton format and all evolutonary operators used n the algorthm. In addton, methods used to encourage populaton dversty are presented. The chapter concludes by demonstratng the operaton of the algorthm through smulaton. Chapter 5 covers the formulaton of and control approach for the drected search problem. The chapter begns wth the formulaton of the problem ncludng representaton of search paths and optmzaton objectves. A method of representng uncertan target trajectores usng Gaussan dstrbutons s then presented, as well a method of calculatng target detecton probablty. The two-tered optmzaton approach usng evolutonary algorthms s then presented. The chapter concludes wth performance results showng the operaton of the algorthm under varous parameter combnatons. The fnal chapter of the thess presents conclusons on the work.

18 6 Chapter Unmanned Survellance Networks An unmanned survellance network s a collecton of unmanned sensor platforms whose efforts are combned to provde nformaton about a regon of nterest. The term can be appled to a large varety of systems, varyng n sze, number of elements, sensor type, platform type, and level of cooperaton between elements. Such networks are typcally used n stuatons were human observaton s mpossble, mpractcal, tedous, expensve, or unsafe. Survellance platforms may be controlled ether autonomously or remotely by human operators. Although ths research focuses on autonomous control of an underwater survellance network, ths chapter presents an overvew of the constructon and control of unmanned survellance networks n general. As nterest n unmanned survellance networks contnues to grow, more and more networks are devsed and control strateges proposed. Ths chapter begns wth a dscusson of typcal objectves of an unmanned survellance network. It then brefly dscusses unmanned vehcles and sensor platforms whch mght be used n an unmanned survellance network. Fnally, a revew of network confguratons and control strateges used by prevous authors s presented.

19 .1 Survellance Network Objectves 7 Although specfc objectves may vary between networks, all unmanned survellance networks share the common objectve of provdng survellance of targets of nterest wthn a gven regon. The nature of ths survellance vares between networks, but generally each node n the network carres some type of sensor whch collects desred nformaton about the target. Examples of gathered nformaton nclude poston, velocty, physcal trats, and other characterstcs such as broadcasted sgnals [1]. Generally, sensor nformaton s fed to a hgh level data fuson process such as a tracker [, 3, 4], classfer [5, 6, 7], or other analyss package such as enemy communcaton structure analyss [1]. Regardless of the purpose for whch sensor data s gathered, the network must be controlled such that survellance of the target s mantaned. Snce the sensors avalable on each node have fnte range and the target may be movng, the network must be reconfgured such that at least one node stays wthn sensor range of the target n order to mantan survellance. The mantenance of target survellance s the purpose of the network and thus, the prmary objectve. In most undersea survellance networks, objectves other than survellance wll lkely play a sgnfcant role as well. Snce most survellance networks are ntended for long-term survellance, network persstence s an mportant objectve. Each node n the network has lmted energy capacty, and rechargng or replacng the power source can be potentally dffcult and tme consumng, partcularly n the case of undersea deployment. In order to obtan the maxmum possble deployment length, each node should consume as lttle energy as possble. In addton, to ensure that as many nodes are avalable for as

20 8 long as possble, energy consumpton should be balanced across the network. Ths helps to ensure that as many nodes as possble are avalable at the end of the deployment perod. In certan scenaros the operaton of the network depends on remanng hdden. In order words, the target must never know t s beng observed nor detect the presence of the network. Dependng on the type of the node and the envronment n whch t s used, a node may need to take delberate measures to avod detecton [8]. If ths s the case, the network must treat counter-detecton as an objectve. Not all survellance network objectves must be drectly related to target survellance. Some networks may have addtonal mssons, such as envronmental montorng. Rather than use a dedcated, fxed sensor network to record samples of envronmental data, as n [9], moble sensor platforms can be used to perform the envronmental montorng task along wth the survellance task, such as the glders n [10]. In systems such as these, plannng vehcle routes to meet envronmental samplng requrements must be a network objectve. Gathered envronmental data may be used strctly for research purposes or may be used to ad the survellance msson. Ths could nclude predctng the future effect of envronmental condtons on network movement and communcatons.. Network Composton The sensor platforms, or nodes, whch form a network, can be broadly classfed nto two categores, fxed nodes and moble nodes. Fxed nodes are those whch have no

21 9 means of self-propulson. Fxed nodes reman n the same poston for the duraton of the network deployment perod. Examples of fxed nodes can be found n ar [11], ground [1], and sea [13] applcatons. Fxed nodes can cost less than many moble nodes and have greater persstence due to ther lack of propulson, navgaton, and other complex systems assocated wth moblty. Unfortunately, due to ther nablty to reposton themselves, a large number of fxed nodes are requred to provde complete coverage of a large survellance area. Moble nodes are those whch have some method of self-propulson. A moble node may use ts propulson system to change ts poston after ntal network deployment. In a system where constant, full-feld coverage s not needed, a small number of moble nodes may be able to effectvely provde the same coverage as a much larger number of fxed nodes. Examples of moble nodes can be found n ar [14], ground [15, 16], and sea [17, 18] applcatons. The addtonal components and complexty requred by a moble node over a fxed node typcally result n a hgher cost. Addtonally, due to the power requrements of most propulson systems, moble nodes typcally have much shorter deployment perods before expendng ther energy reserves. In order to take advantage of the unque benefts of each type of node, the network consdered n ths research uses both fxed and moble nodes, as dscussed n Chapter 3..3 Network Confguraton and Control In recent years, research nterest n the control of unmanned survellance networks has ncreased. Ths s due n large part to technologcal mprovements n the felds of

22 10 autonomous vehcles and wreless sensor platforms. As technologcal mprovements have made an unmanned survellance network a practcal realty, nterest n how to control such a network has grown. Because unmanned survellance networks are a relatvely new feld of actve research, nearly every author presents hs or her own network confguraton along wth a control strategy. Ths secton wll present an overvew of recent approaches, dvded nto three general network confguratons: statc networks, dynamc networks, and responsve networks..3.1 Statc Networks Statc survellance networks are those for whch sensor postons do not change durng the operaton of the network. Such a network may be composed entrely of fxed sensors or may nclude moble sensors whch only change postons between deployment perods. In [19] Khalfa proposes a network composed of fxed sensors, where boundary sensors always reman actve and actvate nearby sensors when a target s detected. A Genetc Algorthm s used to choose fxed sensor postons such that sensor coverage s maxmzed. Another approach for statc network plannng s presented by Ferrar. In [0] Ferrar presents a method for the evaluaton of survellance coverage for a set of fxed sensors, although no optmzaton approach s gven. Unlke the majorty of approaches, whch evaluate total sensor area coverage, Ferrar evaluates the track coverage of a group of sensors. Track coverage s based on the number of potental lnear target trajectores whch pass wthn the sensor range of at least k sensors.

23 11 In order to conserve power, typcally not all sensors n a network reman actve at the same tme. Other authors have dealt wth the problem of schedulng an optmal sequence of sensor actvatons n response to a passng target. In [1] Chhetr proposes a centralzed schedulng method for a network of fxed underwater sensors. The actvaton schedule conssts of a set of actve sensors at each dscrete tme step whch mnmzes trackng error whle satsfyng a maxmum energy usage constrant. To accomplsh ths, the problem s formulated as a bnary programmng problem and solved usng Outer Approxmaton. A dstrbuted schedulng method s presented by Xao n [] and appled to a proposed network of unformly spaced acoustc ampltude sensors. In ths approach, one sensor s desgnated leader at each tme step. At the end of each tme step, the current leader chooses the new leader based on whch sensor gves the hghest probablty of target detecton. Addtonal sensors are actvated at each tme step untl the total detecton probablty reaches a set threshold. Both of the above methods requre complete sensor coverage of the survellance area. Ths requres a large number of fxed sensors or control over where the sensors are placed. In [3] Wang proposes a network composed of both fxed and moble nodes whch provdes for mproved coverage wthout requrng the ablty to exactly place fxed sensors. In ths approach, after fxed sensors are placed, moble nodes are used to fll n coverage gaps. To accomplsh ths, a bddng protocol s used n whch a moble node moves to fll a coverage gap f that gap s larger than the one whch wll be created by the moble node s movement. Ths repeats untl no further mprovements can be made. After ths deployment perod s complete, the moble nodes reman statonary.

24 .3. Dynamc Networks 1 Dynamc sensor networks are those for whch the nodes are n constant moton. Ths allows a smaller number of nodes to cover a much larger area than n a statc network. The coverage propertes of such a network are studed n [4]. In ths paper, the effects of a moble sensor network on area coverage over tme and detecton tme aganst fxed and moble targets are studed. To control the moble nodes, the authors assume a random moblty strategy. In [5], a hybrd network of fxed and moble sensors s proposed. In ths network a small number of moble sensors are used to fll gaps n the coverage of a much larger number of fxed sensors. The authors use a random moton model for the moble nodes and a collaboratve actvaton scheme whch ensures that only one sensor at a tme s actve n a gven regon. The authors show that the addton of moble nodes ncreases the network lfetme and mproves sensor coverage. The analyss of track coverage presented n [0] s extended to moble sensor networks n [6]. In ths paper, Baumgartner and Ferrar gve an objectve functon for optmzng the track coverage of a set of moble sensors n an area of nterest. A soluton s presented for choosng the optmal ntal postons for propulsonless sensor buoys drftng freely n the ocean. The authors show that even for uncontrollable drftng sensor networks, track coverage can be dramatcally mproved by takng ocean currents nto account when choosng ntal sensor postons.

25 .3.3 Responsve Networks 13 Responsve survellance networks are those whch have the ablty to reconfgure n response to the appearance of a target. The ablty to reconfgure the network requres that at least a porton of the nodes be moble. The reconfgurablty of the network allows a small number of nodes to provde coverage for a much larger area than they could otherwse. Although only a porton of the area of nterest s covered at any gven tme, the network s able to reconfgure to provde sensor coverage of targets as they are detected. Such a network requres a control strategy to reallocate nodes n response to the appearance of one or more targets. An example of such a network and control strategy s presented by Ferrar n [7]. In ths network, moble nodes are used to determne tracks (poston, speed, and headng) for targets passng through an area of nterest. Moble nodes are postoned usng three objectves. Frst, nodes should be postoned such that the probablty of detectng as-ofyet undetected tracks s maxmzed. Ths s evaluated usng the track coverage metrc from [0]. Second, partally-observed tracks should be nvestgated. A partally observed track s one of whch less than k detectons have been made. By movng moble nodes near the estmated regon of the partally observed track, t s hoped that more detectons wll be made, mprovng the track. Lastly, the moble nodes should move as lttle as possble. The optmzaton problem s solved usng a centralzed control strategy by dvdng the search area nto a grd and usng the A* graph search algorthm to determne the optmal control acton. Once k detectons of a track have been made, a moble node s dspatched to ntercept the target.

26 14 In contrast to the centralzed strategy above, a dstrbuted control strategy for controllng moble nodes engaged n target trackng s presented by Chung n [8]. In ths control strategy, each moble node uses a gradent-descent technque to mnmze target trackng error. In the event that there are multple targets to track, moble nodes are allocated usng a round robn assgnment procedure. A dfferent dstrbuted control strategy for target trackng usng moble nodes s presented by L n [9]. In ths approach the moble nodes attempt to mantan sensor coverage of the area where the target s lkely located whle mnmzng energy consumpton and mantanng network connectvty. Ths s accomplshed usng a breadth-frst leader-follower algorthm. In ths approach moble nodes are assgned the role of leader n successve rounds n a breadth frst fashon startng wth nodes closest to the target. Each round, the leader moves to provde coverage of the target regon whle mantanng connectvty wth ts prevous leader. In addton, the leader forces ts one-hop followers to move the mnmum requred dstance to mantan connectvty. The authors show ther approach s wthn a scalar factor of the mnmum energy consumpton as formulated. Whle the proposed network and control approach of ths thess belongs to the category of responsve networks, t dffers n many regards from the above approaches. In partcular, the proposed network has much lower node dstrbuton densty and greater node autonomy, as dscussed n Chapter 3.

27 15 Chapter 3 Control Problem Formulaton Due to the large varety of possble underwater survellance network confguratons, the scope of the control problem must be narrowed to focus on a sngle confguraton. Therefore, the focus of ths research s a control approach for a sparsely dstrbuted moble network wth fxed detecton barrer. Ths chapter presents the underwater survellance network to be controlled and begns wth the network objectves whch are to be met. It then presents the assumptons governng the network operaton whch were consdered whle desgnng the control approach. It then llustrates the proposed operaton of the underwater survellance network. Fnally, t presents the abstracton of the overall control problem nto two sub-problems of more manageable scope. 3.1 Survellance Network Objectves Although there are numerous objectves, dscussed n Secton.1, whch may be taken nto account n an underwater survellance network, ths work focused on two. These objectves are provdng maxmum contact survellance coverage and maxmzng network persstence. Provdng contact survellance s the purpose of the underwater survellance network and, as such, s the prmary objectve. The ssue of maxmzng contact

28 16 survellance s two-fold, consstng of ntaton of survellance and mantenance of survellance. Once t s determned that a contact s present, survellance must be ntated. Due to the sparse nature of the network, ths wll mean redeployng a moble node so that the contact s wthn sensor range. In order to mnmze the tme the contact s left unobserved, ths redeployment should take place as quckly as possble. In the lkely event that the exact contact poston s not known, one or more moble nodes must engage n a drected search for the contact. Once the contact s located and survellance begns, ths survellance must be mantaned despte possble speed dfferences between the contact and moble node as well as possble changes n contact course. By mnmzng the tme the contact s left unobserved, the network maxmzes the probablty of detectng any changes n contact course. The second objectve, maxmzng network persstence, supports the man purpose of the network by ensurng that the network s avalable for survellance for as long as possble. As dscussed n Secton.1, the moble nodes have lmted energy capacty and the undersea envronment can make replenshng ths capacty expensve and tme-consumng. Therefore energy conservaton must be a prorty. In order to allow the entre network to be avalable for a long as possble, each node must consume as lttle energy as possble. Ths means mnmzng ndvdual node energy consumpton as well as balancng energy consumpton across the entre network.

29 3. Survellance Network Assumptons 17 Before a control approach can be formulated, the manner n whch the survellance network operates must be defned. The method of operaton for the survellance network s defned n the followng sectons by a prncple concept of operaton, as well as a number of addtonal assumptons Prncple Concept of Operaton The prncple concept of operaton for the network s defned by ts confguraton and ts objectves. Snce the moble nodes are sparsely dstrbuted and provde sensor coverage for only a small porton of the area of nterest, the chances that moble nodes would detect a contact passng through the area of nterest are low. Addtonally, the use of contnual propulson, or even contnuous sensor measurements, would create a sgnfcant energy dran on moble nodes over a long perod of tme. Snce one of the objectves s persstence, ths s unacceptable. Instead, moble nodes n the survellance network reman n the lowest possble power state n the absence of any contacts. Ths allows them to conserve as much power as possble, mprovng persstence. In order to detect contacts enterng the area of nterest, a detecton barrer consstng of fxed nodes s used. Only the nodes n the detecton barrer reman actve at all tmes. As a contact passes through the detecton barrer, reports of detectons from the fxed nodes are fed to a centralzed controller. Ths centralzed controller s responsble for network level control, coordnatng the responses of all nodes. After multple detectons n the barrer have occurred, the centralzed controller assgns survellance

30 18 tasks to one or more moble nodes and brngs these nodes out of ther low-power state to provde survellance. These nodes are not drectly controlled by the centralzed controller, nstead relyng on ntellgent vehcle controllers to perform vehcle routng and sensor management. 3.. Drecton of Contact Approach In order for an effectve detecton barrer to be practcal, some manner of a pror nformaton about the contact s drecton of approach must be known. Ths allows a detecton barrer to be constructed for whch t can be sad, wth some level of confdence, that an unknown contact would need to pass through ths barrer n order to enter the area of nterest. The sze and densty of the detecton barrer depend on the locaton and the level of confdence requred by that stuaton. Such an assumpton s not unreasonable f the survellance network s to be used to provde protecton for a costal regon or bay. In these stuatons, the drectons from whch a possble contact could approach are constraned by the coastlne profle. In some stuatons, a network could be postoned to take advantage of natural bottlenecks n order to reduce the sze of the requred detecton barrer. Examples of possble underwater survellance networks are shown n Fgures 3-1 and 3-. Fgure 3-1 shows an example of an underwater survellance network deployed around a coastal cty or port. Fgure 3- shows an example of a smlar network deployed usng the natural choke pont at the entrance to a bay. In both of these examples, the coastlne profle can be used to constran the drectons from whch a contact could approach, reducng detecton barrer sze.

31 19 Legend: Fxed node Moble node Coastal Cty Fgure 3-1: Example underwater survellance network deployment around coastal cty llustratng restrcted drecton of approach Legend: Fxed node Moble node Fgure 3-: Example underwater survellance network deployment around the entrance to a bay llustratng restrcted drecton of approach

32 3..3 Contact Trajectory 0 Due to the sparse nature of the network, moble nodes wll lkely need to move some non-trval dstance before begnnng survellance. Ths means that n most cases, the nodes must be deployed well n advance of the begnnng of survellance. In order to plan for upcomng node deployments, a projecton of contact poston at any gven future tme must be avalable. For ths reason, contact poston s modeled n ths research usng a pecewse lnear trajectory. Under ths assumpton, the contact poston at any tme t t 0 s defned by n 1 () t p + [ v ( t t )] + v ( t t ) x = n (3.1) 0 1 n 1 = 1 where n s the number of segments, t s the tme the contact completes the th segment, v s the constant contact velocty for the th segment, and p 0 s the ntal poston at ntal tme t 0. A more convenent form s to represent each segment separately, usng () t = p + v ( t t ) t [ t, t ] x (3.) 1 1 for 1 where ( t ) p. (3.3) = x 1 1 Ths contact movement model allows for a fnte number of changes n velocty. However, n the perods between these possble turns and speed changes, the contact moves wth a constant velocty. An example contact trajectory and an estmaton of ths trajectory usng ths model are shown n Fgure 3-3. Although the model s unable to accurately model the trajectory durng the turns, ths s a relatvely small porton of the overall trajectory. Snce the purpose of the model s to project future contact poston,

33 1 modelng the trajectory durng turns s unnecessary, snce only the fnal headng after the turn s needed. An obvous shortcomng of ths model s that f the contact makes long sweepng turns or s contnuously turnng, then the model cannot accurately estmate the trajectory. It s for ths reason that, n ths research, t s assumed that contacts follow a nearly pecewse-lnear trajectory, as n Fgure 3-3, where turns represent a small porton of the overall trajectory Example pecewse-lnear contact trajectory (t 0, p 0 ) Fxed nodes Modeled contact trajectory Actual contact trajectory 60 y [km] (t 1, p 1 ) 0 10 (t, p ) (t 3, p 3 ) x [km] Fgure 3-3: Example contact trajectory and estmaton usng pecewse-lnear model Snce the ntentons of any gven contact are not known, t s assumed that the tme and drecton of possble future turns cannot be predcted. Under ths assumpton, only the current segment of the contact trajectory s needed when plannng moble node responses. Snce the length of the current segment s unknown, t n Equaton 3. s assumed to extend ndefntely, and the contact poston s projected usng

34 ( t) = p + v ( t t ) x (3.4) c c c c where p c s the poston of the contact at the begnnng of the current leg. The contact s located at p c at tme t c, and moves wth constant velocty v c. Ths projected path s referred to as the contact (or target) track. From ths pont forward, the estmated contact trajectory for tmes t tc usng Equaton 3.4 wll be referred to as the contact track. When a contact turn s detected, the current values of t c, v c, and p c are dscarded and replaced wth the new values after the turn Intal Contact Detecton As dscussed n Secton 3..1, durng normal operaton of the network the moble nodes reman n a low-power state n order to conserve energy. Ths places the burden of ntal contact detectons on the detecton barrer. In Secton 3.. t was assumed that nformaton was known about lkely drectons of contact approach. It was also assumed that ths nformaton allowed the detecton barrer to be constructed such that any contacts enterng the area of nterest must frst pass through the detecton barrer. The detectons whch occur as the contact passes through the detecton barrer are used to form an ntal estmate of the contact track. The qualty of the track estmate could depend on a number of parameters ncludng number of detectons, postonng of fxed nodes, qualty and type of fxed node sensors, speed of contact, angle of contact approach, and acoustc sgnature of the contact. Snce the qualty, type, and postons of the fxed nodes are statc after ntal deployment, these ssues are not consdered by the

35 3 control problem studed here. It s, however, assumed that after the contact passes through the detecton barrer, an estmated ntal track s produced. Ths estmated track represents possble contact tracks, and a confdence n each possble track. Detals about the specfc representaton for uncertan tracks used n ths work are dscussed n Secton Dmensonalty Snce the majorty of the applcatons consdered for underwater survellance networks occur near the shorelne or a choke pont, the ocean depth n most network operaton areas wll be relatvely shallow. Because of ths, the horzontal area consdered wll be much larger than the vertcal area. Based on ths, t s assumed that depth has the smallest role n plannng and s therefore gnored. Under ths assumpton all plannng and network control takes place n the two horzontal dmensons (x and y). Moble nodes are assumed to be responsble for managng ther own depth. Lack of consderaton for depth offers a number of advantages. Frst, t results n a greatly reduced soluton space snce plannng occurs n two, rather than three, dmensons. Second, t removes the multple depth constrants to whch the moble nodes are subject. Each moble node has a maxmum depth floor, whch may be dfferent for each node. In addton to maxmum operatonal depths, the bathymetry 1 of the operaton area would have to be consdered. The addton of all of these factors could add consderable complexty to the problem. 1 The depth of the ocean floor below the surface

36 3..6 Communcatons 4 Many autonomous underwater vehcles use acoustc modems for communcaton. Whle these acoustc modems may be used to form hghly connected underwater communcaton networks [30], ths may not be possble n the survellance network consdered here. Even under deal condtons, acoustc modems have lmted range [31]. Due to the large network sze and sparse node dstrbuton n the consdered network, the separaton between nodes s lkely to exceed the range of the acoustc modem. Due to ths fact, t s assumed that no nter-node communcaton takes place n the network. Despte the lack relable of nter-node communcaton, t s assumed that a system s n place to allow the fxed and moble nodes to communcate wth the central controller. A number of such systems usng acoustc modems currently exst, ncludng gateway buoys and glders. A gateway buoy [3, 33, 34] s desgned to float at the surface and relay acoustc communcatons to a land-based staton usng a satellte lnk. A glder s an AUV whch adjusts ts buoyancy to propel tself through the water [35]. Ths causes a glder to be extremely power effcent, makng t practcal for t to reman n constant moton. In addton to uses n envronmental samplng and survellance, glders have been proposed for use as moble communcaton gateways [36]. Lke gateway buoys, the glders act as a relay between acoustc and satellte communcatons. Due to ther moblty, many fewer glders than buoys would be necessary to cover a gven area. However, glders would ntroduce a communcatons delay as they must come to the surface n order to establsh satellte communcaton. Regardless of the method used, t s

37 assumed that some means of communcaton between nodes n the network and the central controller s avalable, although not necessarly contnuously Autonomy Because the moble nodes n the survellance network are autonomous, t s assumed that only hgh-level control s needed. Rather than exert drect control over the vehcles, the central controller ssues wayponts, nstructng the vehcles to move to gven postons at gven tmes. At each waypont, the central controller nforms the vehcle how t should behave. Ths ncludes: engagng sensors and actvely searchng for a contact, begnnng survellance of a contact, or restng n a low-power state. Once a moble node s nstructed to begn survellance, t autonomously chooses ts pursut path untl nstructed to stop by the central controller. The subject of choosng the optmal observer path for target moton analyss has already been the subject of much research [37, 38]. Under the assumpton of moble node autonomy, the central controller s responsble for placng the correct vehcles n the correct postons at the correct tmes. By mantanng a global network vew, the central controller s able to select and poston nodes such that objectves are met across the entre network Statons Under the prncple operatonal concept of the network, moble nodes reman n a low-power state when no contacts are present. Snce, for most AUVs, propulson s the

38 6 system whch consumes the most power, any low-power mode must, almost certanly, requre the vehcle to use ts propulson system as lttle as possble. Ths wll leave the vehcle drftng, subject to ocean currents. If all moble nodes are allowed to drft for an ndefnte amount of tme, nodes wll eventually drft away from ther ntal poston, leavng gaps n the network. An example of ths s shown n Fgures 3-4 and 3-5. Gven the ntal node postons and ocean currents shown n Fgure 3-4, the passage of suffcent tme wll cause the nodes to drft to the postons shown n Fgure 3-5. The change n node postons leaves a regon, approxmately marked wth red, where moble nodes may not be able to ntercept a possble contact. Ths compromses network survellance coverage and therefore must not be allowed. 100 Uncontrolled drftng wth ocean current y [km] Fxed nodes Moble nodes Ocean current x [km] Fgure 3-4: Moble node postons before uncontrolled drftng, showng ocean currents

39 7 100 Uncontrolled drftng wth ocean current y [km] Fxed nodes Moble nodes Ocean current Drft trajectory Uncovered regon x [km] Fgure 3-5: Moble node postons after uncontrolled drftng, showng exposed regon In order to ensure that network coverage s not compromsed by ocean currents, moble nodes are assgned statons whle not provdng survellance. A staton s a small area where the moble node s to reman untl gven new nstructons. Whle holdng staton, a moble node should drft freely as often as possble, so long as t remans wthn a gven dstance of ts assgned staton poston. If a node drfts beyond the allowed dstance, t wll use ts propulson system to return to staton. Ths allows the nodes to reman n a low-power state the majorty of the tme, whle mantanng network coverage. Although nodes are assgned to an ntal staton, staton assgnments can be changed durng the operaton of the network. Ths allows moble nodes to swap statons f t s benefcal for the network. Despte ther ablty to swap statons, moble node staton assgnments must be unque. That s, only one node may occupy each staton at a gven tme. Allowng two nodes to occupy the same staton goes aganst the motvaton

40 for usng statons, mantanng network coverage. An example set of statons s shown n Fgure Moble nodes and staton regons Fxed nodes Moble nodes Staton y [km] x [km] Fgure 3-6: Moble nodes and staton assgnments 3.3 Survellance Network Operaton Ths secton wll present an outlne of the operaton of the proposed underwater survellance network. The network operaton s based on the network confguraton, objectves, prncple of operaton, and assumptons dscussed n the prevous sectons. The network operaton s dvded nto four phases: steady state, ntal detecton, drected search (estmated track), and allocaton (accurate track).

41 9 When no contact s present n the area of nterest, the network operates n the steady state phase of operaton. In ths phase, the network attempts to conserve as much power as possble whle retanng network coverage and watchng for new contacts. Ths s accomplshed by placng the moble nodes n a low-power mode, whle leavng the fxed nodes to watch for contacts. In the low-power mode, moble nodes deactvate ther sensors and drft wth the ocean currents. In order to prevent drftng from causng openngs n the network, moble nodes are assgned statons to whch they must stay near. The steady state phase ends when a contact enters the area of nterest, passng through the detecton barrer. When the frst fxed node detects the contact, the network enters the ntal detecton phase. In the ntal detecton phase, the central controller collects detecton reports from fxed nodes as the contact passes through the detecton barrer. As these detecton reports arrve, they are fed to a tracker whch generates an estmated track for the contact. Once ths track s generated, the network enters the search phase. In the search phase, moble nodes are dspatched to search for the contact, based on the estmated track. The central controller chooses one or more moble nodes to cooperatvely search for the contact. The central controller generates approxmate search paths for each of these nodes usng nformaton provded by the estmated track. If one of the moble nodes fnds the contact, t begns followng the contact and recordng addtonal sensor readngs. Once a suffcent number of readngs are taken, such that an accurate track s obtaned, the network enters the allocaton phase. In the allocaton phase, the network allocates one or more moble nodes to the survellance task. The central controller determnes whch nodes wll be used for

42 30 survellance and over what ntervals they wll provde survellance. The controller chooses allocatons whch optmze the network objectves. If one of the moble nodes detects a change n contact velocty, a new track s generated and the assets are reallocated by the central controller. In addton to allocatng survellance wndows durng the allocaton phase, the central controller also assgns moble nodes to statons. Once each node fnshes ts survellance task, t returns to ts assgned staton. When the contact exts the area of nterest, the network returns to the steady state phase. 3.4 Problem Defnton and Abstracton Under the operaton process defned n Secton 3.3, the assumptons defned n Secton 3., and the objectves defned n Secton 3.1; the central controller s responsble for two tasks. The frst s to generate approxmate search paths for moble nodes to follow n a drected search. It s mportant for the central controller to perform ths task because t elmnates redundancy. Moble nodes plannng ther own search paths would lkely search the same area as other nodes. In addton, the central controller receves all contact detectons from the detecton barrer and has the most up-to-date track nformaton untl the frst moble node detects the contact, at whch pont the drected search s over. The second control task s that of allocatng nodes to the survellance task. Once an accurate track s determned, the centralzed controller must choose whch moble nodes are used to provde survellance. In addton, t must determne over what nterval each of these nodes s used to provde survellance.

43 31 Consderng both tasks as a sngle problem would have made the problem ntractable. Instead, t was decded to abstract the overall problem as two smaller subproblems. The drected search problem consders the task of choosng moble node search paths. The asset allocaton problem consders the task of allocatng nodes to the survellance task Drected Search The purpose of the drected search problem s to choose moble node search paths n order to best locate the contact. The search paths are chosen such that the maxmzaton of contact detecton probablty s balanced aganst the mnmzaton of moble node energy consumpton. The probablty that a gven set of search paths wll result n the detecton of the contact s determned usng the estmated track generated as the contact passes through the detecton barrer. Ths estmated track takes the form of probablty densty functons (PDFs) for the poston and velocty of the contact. Usng a constant velocty assumpton, these functons are used to determne the probablty that a set of search paths wll result n the detecton of the contact. Gven the autonomous nature of the moble nodes and the lmted communcaton avalable, t s not possble, nor would t be practcal, to specfy exact search paths for each node. Instead, the central controller specfes approxmate search paths. These search paths consst of a seres of wayponts and correspondng tmes. The moble node

44 3 should reach each waypont by the gven tme. The responsblty of choosng the actual path taken between wayponts s left to the AUV. An example of a drected search s shown n Fgure 3-7. The red lne ndcates the actual contact track, whle the broken lnes represent the bounds on the uncertan track estmate. The dashed lne represents the approxmate bounds on the uncertan ntal poston. The dash-dot lne represents the approxmate bounds on the uncertan future poston due to uncertan ntal poston and uncertan velocty. A possble drected search soluton, consstng of a set of wayponts for each node nvolved n the search, s represented by the sold dots. Example trajectores for nodes followng these wayponts are shown wth sold lnes. If the nodes have not located the contact by the tme they reach the end of ther search paths, they return to ther statons. At ths pont, further searchng would not be effcent as the contact poston s extremely uncertan. Contnung the search would requre expendng large amounts of energy wth a low probablty of detectng the contact. Ths would be contrary to the persstence objectve of the network. The proposed approach for generatng drected search paths s presented n Chapter 5.

45 Example drected search y [km] Fxed nodes Moble nodes Statons Actual track Uncertanty n poston Uncertanty n velocty Search path waypont Search path x [km] Fgure 3-7: Example drected search showng wayponts and possble moble node paths 3.4. Asset Allocaton Due to the sze of the area of nterest and the separaton between nodes, t s not practcal for every node to respond to each contact. The survellance task must be dvded between a subset of the network populaton such that survellance s mantaned whle mnmzng power consumpton. Ths s the purpose of the asset allocaton problem, to choose moble nodes and survellance areas for those nodes such that network objectves are optmzed. Ths can only be performed f a suffcently accurate track s known for the contact. Because of ths, asset allocaton can only take place after the drected search s complete and an accurate track s determned. If the drected search s not successful, asset allocaton does not take place, as no relable nformaton exsts wth whch to drect

46 34 the response of the moble nodes. In the event of an unsuccessful drected search, all moble nodes return to ther statons. If the search does result n an accurate track, asset allocaton s used to drect the survellance response of the nodes. Due to the autonomous nature of the moble nodes, assgnng coverage ntervals to nodes conssts of specfyng the postons to begn and end survellance. To begn survellance, the node s nstructed to move a specfed poston and wat for the contact. Ths poston s referred to as the ntercept poston. Once the contact s detected, the node s to autonomously pursue t. The node should contnue to pursue the contact, provdng survellance, untl t reaches a break-off poston specfed by the central controller. Once ths poston s reached, the moble node ends ts survellance of the contact and returns to ts assgned staton. Nodes whch are not chosen to provde contact survellance move drectly to ther assgned statons. An example survellance response, as drected by an asset allocaton soluton, s shown n Fgures Fgure 3-8 shows the network condtons before asset allocaton, ncludng node and contact postons as well as the contact track. Black dots ndcate the ntercept and break-off postons specfed by a possble asset allocaton soluton. In the soluton presented here, nodes 1,, 5, and 6 are chosen to be used for survellance. In fgure 3-9, nodes 5 and 6 begn movng to ther ntercept postons. The contact s stll a great dstance from nodes 1 and, whch wat to move to ther ntercept postons. Once node 6 detects the contact t begns pursung t, as shown n Fgure When node 6 reaches the specfed break-off pont, t begns to return to ts staton. At ths pont, survellance s handed off to node 5. Meanwhle, node begns to move to ts

47 35 ntercept pont. Ths process s repeated for the remanng nodes whch are nvolved n survellance. Resultng moble node trajectores as the contact exts the area of nterest are shown n Fgure After the contact exts the area of nterest, the fnal moble node returns to ts staton. The proposed approach to choosng the asset allocaton soluton s presented n Chapter Survellance response drected by asset allocaton y [km] Fxed nodes Moble nodes Statons Contact track Contact poston Soluton pont x [km] Fgure 3-8: Intal condtons for example survellance response as drected by asset allocaton soluton, showng ntercept and break-off ponts

48 Survellance response drected by asset allocaton y [km] x [km] Fgure 3-9: Example asset allocaton response wth nodes movng to ntercept postons 100 Survellance response drected by asset allocaton y [km] x [km] Fgure 3-10: Example asset allocaton response showng survellance handoff

49 Survellance response drected by asset allocaton y [km] x [km] Fgure 3-11: Example asset allocaton response showng node trajectores as contact exts area of nterest

50 38 Chapter 4 Asset Allocaton As descrbed n Secton 3.4., the asset allocaton problem s that of allocatng avalable nodes n an unmanned underwater survellance network to the task of target survellance. Gven a known target ntal poston and velocty, nodes must be chosen and survellance regons assgned such that a gven set of objectves are optmzed. Ths chapter presents a method for the generaton of solutons to asset allocaton problems. Ths chapter begns wth a detaled formulaton of the asset allocaton problem, ncludng the representaton of canddate solutons, boundares and constrants on the values of these solutons, and an objectve functon aganst whch these solutons are optmzed. Possble strateges for mxed nteger optmzaton are then dscussed, ncludng the chosen method, Evolutonary Strateges. The applcaton of Evolutonary Strateges to the asset allocaton problem s then presented, coverng all evolutonary operators and heurstcs used n the presented algorthm. Fnally, results are presented demonstratng the effectveness of the allocaton approach. 4.1 Problem Formulaton As dscussed n Chapter 3, the central controller n the survellance network s responsble for hgh-level plannng rather than drect AUV control. As presented n Secton 3.4, the asset allocaton problem can be formulated as a mult-objectve

51 39 optmzaton problem. Ths secton wll present the formulaton of ths optmzaton problem. Ths ncludes the representaton of canddate solutons, the objectves whch are to be optmzed, and constrants and boundares of the optmzaton problem Soluton Representaton As the asset allocaton problem s presented n Secton 3.4., four parameters are requred to formulate a soluton: actve status, ntercept poston, break-off poston, and staton assgnment. Two of these parameters must be defned for each moble node. The frst of these parameters s the actve status, whch ndcates whether a node s actve or nactve n the current soluton. An actve node s one whch s to provde contact survellance. An actve node moves to a specfed ntercept poston and begns followng the contact whle provdng survellance. Once the node reaches a specfed break-off poston, t ends survellance and returns to ts assgned staton. An nactve node s one whch provdes no survellance and moves drectly to ts assgned staton. Staton assgnments must be unque and one must be made for every moble node. The remanng two parameters, ntercept poston and break-off poston, must be defned for every actve node n the current soluton. Gven the above parameters, an effcent representaton s needed for each n order to completely defne the soluton. The actve status parameter can easly be represented usng a bnary flag for each node. If the flag s set for a gven node, that node s actve n the current soluton. Snce the quantty and postons of the statons do not change durng operaton, the staton assgnments can easly be represented for each node

52 40 usng the ndex of the assgned staton. Agan, each node must be assgned to a unque staton. The remanng parameters, ntercept poston and break-off poston, each consst of a par of (x,y) coordnates. However, a more effcent representaton exsts due to the way the parameters are defned. Because these parameters specfy postons where the node meets or departs the contact, these ponts must le on the trajectory of both the moble node and the contact. Therefore, the ntercept and break-off ponts can be specfed as ponts along the contact trajectory. Because the contact trajectory s lnear, only one parameter s necessary to defne each pont. Usng the contact trajectory, each ntercept and break-off pont can be specfed usng a sngle tme value. Ths tme value, along wth the contact velocty and ntal poston, defnes a unque poston n (x,y) space (represented by the vector x). Usng ths representaton, the ntercept poston for node can be calculated usng nt c ( t t ) = x ( t ) + ( t t ) x = x v, (4.1) ntercept c c ntercept where x c (t) s the contact poston at tme t and v c s the contact velocty. Smlarly, the break-off poston for node can be calculated usng brk These defntons can be smplfed by representng ntercept and break-off ponts usng relatve tmes, c ( t t ) = x ( t ) + v ( t t ) = xc break off c c c c break off c x. (4.) t t nt brk = t = t ntercept break off c c c tc, (4.3) t and referrng to x c (t c ) as x c, resultng n

53 x x nt brk = x c = x c + v ( tnt ) ( t ) Usng the above representatons, the parameters can be combned nto a sngle potental soluton, as llustrated n Table 4-1 for an example wth 6 moble nodes. In ths example, nodes, 5, and 6 are actve and provde survellance, whle nodes 1, 3, and 4 reman at ther statons. A graphcal equvalent of ths soluton s shown n Fgure 4-1. In ths example, the nodes begn at statons wth ndces dentcal to ther own. Snce the resultng staton assgnments are dentcal, each node returns to the staton from whch t started. c + v c brk. (4.4) 41 Table 4-1: Asset allocaton soluton correspondng to Fgure 4-1 Moble node # Actve Intercept tme -- t nt t nt Break off tme -- t brk t brk Staton t nt 6 t brk

54 4 100 Example asset allocaton soluton (x,ynt nt ) (x c,y c ) y [km] (x,ybrk brk ) (x,ynt nt ) (x,ynt nt ) 5 5 (x,ybrk brk ) (x,ybrk brk ) x [km] Fxed node Moble node Staton Track Contact Intercept segment Follow segment Return segment Intercept pont Break-off pont Fgure 4-1: Example asset allocaton soluton correspondng to Table 4-1

55 4.1. Constrants 43 Not all sets of parameters of the form presented n the prevous secton represent vald solutons. Solutons must satsfy three constrants n order to be consdered vald. The frst constrant, motvated by the defnton of statons n Secton 3..8, s the staton unqueness constrant. Ths states that all staton assgnments must be unque. Ths constrant s smply enforced by ensurng that the staton ndex assgned to each node s not assgned to any other node n the gven soluton. It s strctly enforced, any soluton whch volates ths constrant s not allowed. The second constrant, motvated by the defnton of ntercept and break-off tme, s the orderng constrant. Ths states that the break-off tme must be greater than the ntercept tme. A soluton whch volated ths constrant for any node would not correspond to a physcal response snce survellance would end before t started. Ths constrant s enforced by a smple comparson for each node. Lke the unqueness constrant t s strctly enforced, any soluton whch volates ths constrant s not allowed. The fnal constrant, motvated by the physcal lmtatons of the moble nodes, s the speed constrant. Ths constrant only apples n the case where the contact travels faster than a moble node s maxmum pursut speed. A moble node s pursut speed may be lower than ts maxmum speed, dependng on the pursut trajectory used by the node. In the event that ths speed s less than the contact s speed, the moble node cannot provde survellance for an unlmted length of tme. The maxmum survellance duraton s lmted by the sensor range of the moble node and the speed dfferental between the node and contact. An example of ths lmted survellance s shown n Fgure 4-. Fgure

56 44 4- (a) shows the contact and node postons at the begnnng of survellance, t = t begn. Assume that at the begnnng of survellance, the node and contact are located at the same poston. Due to the speed advantage of the contact, t wll move away from the node, eventually leavng the node s sensor area. The pont where the contact leaves the node s sensor area and survellance must end s shown n Fgure 4- (b), at tme t = t end. Begnnng of survellance (t = t begn ) End of survellance (t = t end ) x n (t begn ) x c (t begn ) r detect 0 x 0 x n (t end ) y [km] -5 y [km] -5 x c (t end ) r detect x [km] (a) Moble node x [km] Moble node detecton radus Contact track Contact Fgure 4-: Begnnng (a) and end (b) of lmted survellance wndow (b) The example n Fgure 4- presents the maxmum length survellance wndow for a contact wth a speed advantage. In order to check f a soluton meets ths constrant, ths maxmum wndow length must be calculated for each node wth a speed dsadvantage. At

57 45 the begnnng of survellance, the contact poston (x c ) and the moble node poston (x n ) are dentcal, wth x c (t begn ) = x n (t begn ) = x 0. The contact leaves the node s sensor radus and survellance ends at t = t end = t begn + t wndow, where t wndow s the length of the maxmum possble survellance wndow. At the end of survellance, the moble node and contact postons are x ( t n x ( t c end end ) = x ) = x v + v n c ( tend tbegn ) = x0 + vn( twndow ) ( t t ) = x + v ( t ) end begn wndow At the pont where survellance ends, the contact s at the edge of the node s detecton radus. Therefore, the dstance between the contact and node s gven by 0 c. (4.5) x ( t c end ) x ( t n end ) = r = = detect [ x0 + vc ( twndow )] [ x0 + vn( twndow )] ( v c v n ) t wndow, (4.6) where v c s the contact velocty, v n s the average pursut velocty of the moble node, and r detect s the detecton radus of the node. Snce the contact and node are movng n the same drecton, v c s a scaled verson of v n. Therefore, Equaton 4.6 can be expressed as c end ( vc vn ) twndow rdetect x ( t ) x ( t ) = = (4.7) Rewrtng, the maxmum survellance wndow s expressed as n t end r = detect wndow vc v (4.8) n for nodes wth v < v. Snce every node wth a speed dsadvantage must be subject to n c a maxmum length survellance wndow, the speed constrant can be expressed as rdetect ( t t ) { N : v > v } brk nt v c v n 1 c n, (4.9)

58 where N s the number of moble nodes. Ths constrant s enforced as a soft constrant, as dscussed n Secton Boundares Before a soluton can be generated, boundary values must be establshed so t can be determned f a gven soluton s vald. These boundary values must be determned for every soluton parameter. The boundary values for the actve status and staton assgnment are easly determned. The actve status parameter must be a bnary flag, { 0,1} actve. The staton assgnment must be an nteger, wth 1 staton N s, where N s s the number of avalable statons. In ths work, the number of statons wll be equal to the number of moble nodes, N. The ntercept and break-off tmes must be contnuous varables, subject to multple boundares. These boundares are defned by the area of nterest, moble node operatonal areas, and mnmum ntercept tmes, dscussed n the followng sectons Area of Interest Boundares The frst boundary s a result of the sze of the area of nterest. Snce the nodes are to reman n the area of nterest at all tmes, the bounds on ntercept and break-off tmes must reflect ths requrement. The boundary of ths area of nterest s defned by an N v pont polygon. Ths polygon can take any shape, so long as t s convex. A polygon s sad to be convex f the removal of any pont results n a subset of the orgnal polygon.

59 47 That s, f the N v pont polygon s defned as the set P and P s the set defned by the polygon P wth pont removed, then P s convex f and only f {, } P P 1 L,. (4.10) N v An example of such a polygon s shown n Fgure 4-3. The x v correspond to the postons of the vertces of the polygon whle x c s the current poston of the contact x 1 v Area of nterest boundary x Nv v x Nv-1 v x c x Nv- v 60 y [km] x 3 v x v x [km] Fgure 4-3: Example area of nterest bounded by a convex polygon Snce the contact must be nsde the area of nterest before a track can be determned, the lower bound on ntercept and break-off tme s zero. The upper bound s the tme when the contact exts the area of nterest. Ths can be determned by calculatng Usng relatve ntercept and break-off tme parameters, zero corresponds to the current tme.

60 48 the tme at whch the contact wll ntersect the boundary polygon. The segment of the polygon whch ntersects the track s determned by comparng the headng along whch the contact s travelng ( ( v ), ) ψ = mod π, (4.11) c to the angle each vertex makes wth the contact s poston, ( ( x x ), ) φ = mod π. (4.1) The polygon segment whch ntersects the track s formed by the vertces x v and x j v between whch the track passes. These vertces satsfy the condtons: v c φ ψ, (4.13) φ j ( j,( N ) ) 1 mod =, (4.14) v { 1,,, 1 }, j L N v, N v. (4.15) Once the vertces are found, the tme requred for the contact to reach the resultng segment s calculated usng t ext = = j ( x v x c ) ( x v x v ) j v c ( x v x v ) j [( x v x c ) ( x v x v )] j [ v ( x x )] c The ext tme, t ext, represents the tme t wll take the contact to reach the edge of the area of nterest from ts ntal poston. Snce no moble node may ext the area of nterest, ntercept and break-off tmes for all nodes must be less than the ext tme. v v. (4.16)

61 49 Therefore the boundary whch ensures that moble nodes reman n the area of nterest s defned as t j nt B, t AOI j brk = B AOI { t R : 0 t t } ext. (4.17) Operaton Area Boundares Whle all nodes are subject to the area of nterest boundary, some nodes may have addtonal boundares mposed by lmted operaton areas. These lmted operaton areas could be the result of a number of factors. One example s the need to mantan a certan depth, requrng the node to avod shallow regons. Another example entals a node remanng n a gven regon n order to mantan communcatons. Regardless of the motvaton, f a node has a lmted operaton area, t s represented by a convex polygon POP N N v v = x R : x = λ jxv, 0 λ 1, = 1 j j λ j, (4.18) j= 1 j= 1 where x vj are the N v vertces of the boundary polygon for node. The defnton of the operaton area requres that the condton n ( t) POP 1 N v x (4.19) be met at all tmes. For nodes whch have operaton areas defned, ther operaton areas are used to calculate boundares on ntercept and break-off tme for that node. An example of moble node operaton boundares s shown n Fgure 4-4. In ths example, nodes 1,, and 3 have lmted operaton areas, whle node 4 has none.

62 Operaton area boundares Area of nterest boundary P OP x n 3 x 1 v 3 x v 3 x n x Nv 3-1 v 3 x Nv 3 v 3 y [km] x 1 v 1 x v 1 x n 1 1 P OP x Nv v x 1 v x Nv -1 v x n 0 P OP 10 x Nv 1 v 1 x Nv 1-1 v x [km] Fgure 4-4: Example moble node operaton areas x v There are three condtons whch determne the type of boundary enforced by an operaton area for each node. The smplest condton s that n whch a moble node has no specfed operaton area other than the area of nterest. In ths case, the operaton area boundary for node s gven by In the event that a node does have an operaton area specfed, the type of boundary s determned by the ntal track poston. If the ntal track poston s wthn the operaton area of node, x c P OP t, nt tbrk BOP. (4.0) B = OP { t R}, then the boundary values for ntercept and break-off tme are

63 calculated n the same manner as for the area of nterest boundares. That s, Equatons are used to calculate the ext tme, t ext, of the contact from the operaton area of node. The boundary set s then defned as Ths approach must be expanded for the case where a node has an operaton area defned and x c P OP t, t B nt brk OP = P { t R : 0 t t }, n other words, when the ntal track poston s not wthn the operaton area. In ths case, the operaton area defnes both a lower and an upper bound on the ntercept and break-off tme. If the contact s on course to pass through the operaton area, there wll be two polygon segments whch satsfy Equatons These segments correspond to the entrance and ext of the contact from the operaton area. Each of these segments s used to generate a soluton to Equaton The lesser of these two tmes s the tme at whch the contact enters the operaton area of node, t enter, the other s the tme at whch the contact exts the operaton area, t ext. The boundary set for the operaton area of node s then defned as t, t B nt brk OP = B If the contact s not on course to pass through the operaton area of node, then there are no segments whch satsfy Equatons If ths s the case, node s unable to respond to the contact and the operaton area boundary set s defned as OP OP ext { t R : t t t } t,t B nt brk OP enter B = { } OP ext. (4.1). (4.). (4.3) 51

64 Mnmum Intercept Tmes 5 The fnal boundary s a result of problem geometry and vehcle parameters: the mnmum ntercept tme boundary. The mnmum ntercept tme for each node places a lower bound on the ntercept and break-off tmes. The mnmum ntercept tme represents the shortest possble length of tme t could take a node to reach the contact, f t started movng mmedately. The mnmum ntercept tme corresponds to the optmal ntercept path n whch the node travels n a straght lne at maxmum speed. The headng whch causes ths trajectory to ntersect the contact trajectory gves the mnmum ntercept tme. In addton to the speeds of the contact and moble node, ths calculaton must take the ocean currents nto account. A dagram, llustratng the optmal ntercept trajectory, s shown n Fgure 4-5. (x c,y c ) v c t (x,y ) v o t v n t v n t + v o t (x a,y a ) Fgure 4-5: Optmal moble node ntercept trajectory

65 Table 4- lsts the parameters n the ntercept problem whch are known and those whch are unknown. The parameters lsted are for only a sngle node. 53 Table 4-: Optmal ntercept problem parameters for a sngle node Known parameters v c = [v cx v cy ] Contact velocty v o = [v ox v oy ] Current velocty v n Maxmum moble node speed x c = (x c, y c ) Contact poston (at tme t = 0) x a = (x a, y a ) Moble node poston (at tme t = 0) Unknown parameters φ Optmal moble node ntercept headng t Optmal (mnmum) ntercept tme x = (x, y ) Optmal ntercept poston At the optmal (mnmum) ntercept tme, the contact wll be located at the optmal ntercept poston. Snce all known nformaton about contact velocty s gathered by observaton, the contact velocty s consdered to nclude the effects of currents. Therefore, the contact poston at the optmal ntercept tme s gven by x = x c + v c t or equvalently, x = xc + vc t x. (4.4) y = y + v t c c y Smlarly, by the defnton of the optmal ntercept tme and poston, at the optmal ntercept tme the moble node wll be located at the optmal ntercept poston. Snce the maxmum moble node speed s known ndependent of ocean currents, ther effects must be ncluded. Therefore, the node poston at the optmal ntercept tme s gven by x = x a + v n t + v o t or equvalently, x = x + a y = y + a ( vn cosφ) t + vo t x ( vn snφ ) t + v t o y. (4.5)

66 54 Defnng and solvng Equatons 4.4 and 4.5 for t yelds Ths quadratc equaton can then be solved for the two possble ntercept tmes, t 1.and t. The mnmum ntercept tme s the mnmum postve root. If the roots are negatve or magnary, then the node s never able to ntercept the contact. If the roots are real and postve but greater than the contact ext tme, then the node cannot ntercept the contact wthn the area of nterest. Therefore, the mnmum ntercept tme boundary s defned for node j as where t 1 j and t j are the roots of the mnmum ntercept problem for node j and Thus, the mnmum ntercept tme establshes lower bounds on the value of ntercept and break-off tme parameters. a c y a c x y y x x = = δ δ (4.6) ( ) ( ) ( ) 0 = y x o y o x c y c x o c o c n c c o o t v v v v t v v v v v v v v v y x y x y y x x y x y x δ δ δ δ δ δ. (4.7) ( ) R = otherwse T f T t t B B t t j j j ntcp j ntcp j brk j nt mn :,, (4.8) { } { } { } ext j j j j j t t t t k t T k k k k < > = =, 0 0, Im, 1, :. (4.9)

67 Combned Boundares 55 Snce the ntercept and break-off tmes for each node must be subject to the area of nterest, operaton area, and mnmum ntercept boundares, the overall boundares on these parameters are defned by t nt B T, t brk = B B AOI T B OP B ntcp. (4.30) If BT = for a gven node, that node s unable to provde contact survellance. Any node whch meets ths condton s removed from further consderaton n the current soluton. It wll be assgned to reman at ts current staton untl t s agan able to partcpate Optmzaton Objectves In order to dentfy an optmal soluton to the asset allocaton problem, an objectve functon must be defned. Ths objectve functon s used to evaluate a soluton and assgn a scalar value whch s used to determne the superorty of a gven soluton over another. The objectve functon s based on the objectves of the survellance network, dscussed n Secton 3.1, as well as the volaton of constrants, dscussed n Secton The objectves are the mnmzaton of contact free tme, the mnmzaton of energy consumpton, and the even dstrbuton of energy costs. Free tme s the amount of tme t s predcted that the contact wll spend n the area of nterest whle not under survellance by any moble nodes. A soluton whch mnmzes free tme and energy

68 56 consumpton, accordng to the manner prescrbed n the objectve functon, s the optmal soluton. The objectve cost of free tme s measured by the rato of the tme the contact spends n the area of nterest unobserved to the total tme t spends n the area of nterest. Due to the autonomous nature of the moble nodes, the amount of tme the contact wll spend under survellance cannot be exactly predcted. However, due to the large tme frame under consderaton, ths uncertanty wll cause relatvely lttle varaton n the survellance tme from that gven by the ntercept and break-off tmes defned for each actve node. Therefore, the nterval over whch node s consdered to provde contact survellance s expressed as [t nt, t brk ]. Under ths defnton survellance s provded durng all tmes exstng on the set S U[ t nt t ] brk =,, (4.31) A where A s the set of ndces of actve nodes n the soluton whch s beng evaluated. The free tme rato s determned by frst defnng a coverage functon, 1 f coverage () t = 0 f f t S, (4.3) t S then evaluatng c freetme t 0 = ext f coverage t ext () t dt 1. (4.33)

69 57 Usng ths formula, the free tme cost s equal to 0 when survellance coverage covers 100% of the possble wndow,.e. when S = [0, t ext ]. Smlarly, the free tme cost s 1 when there s no survellance coverage,.e. when S =. The objectve cost of energy consumpton s measured n two parts. The frst part consders the amount of energy whch wll be consumed by the proposed soluton, whle the second part consders the degree to whch the energy consumpton s balanced across the network. Both of these metrcs must be consdered n order to ensure that all network objectves are taken nto account. The requred energy consumpton cost s calculated usng a weghted average of the runtme for each node. The runtme for a node s the amount of tme that node wll spend travelng under ts own power n the response correspondng to the soluton beng evaluated. The calculaton of runtme for a gven soluton s dscussed n Secton Usng ths model, energy consumpton has a lnear relatonshp to runtme. Usng the weghted average, the energy consumpton cost s calculated usng c energy = N ( f ) ( wreserve )( runtme ) reman = 1 (4.34) where f reman s the fracton of ntal energy reserves for node whch wll reman after the soluton s executed. The weghtng functon, w reserve, weghts the energy cost based on the fracton of energy reserves remanng. For each node, gven the proposed runtme, runtme, the number of hours of energy remanng, hrsreman, and the ntal hours of energy reserves, hrsreman nt, the fracton of energy remanng s gven by f hrsreman runtme reman =. nt (4.35) hrsreman

70 58 The weghtng functons on each term are used to ncrease the energy cost for each node as t approaches zero remanng energy. Snce a node s of no use when ts energy reserves are depleted, the weght ncreases quckly as energy reserves approach zero. Ths dscourages solutons from beng chosen whch wll completely deplete the energy of a node unless completely necessary. The weghtng functon s equal to one, addng no penalty, untl the fracton of energy remanng drops below a fxed low-power threshold, LP. After ths pont, the weght ncreases exponentally, accordng to w reserve ( f ) reman MP e = λ freman ( c ) The parameter MP determnes the maxmum penalty weght whch may be assgned when the remanng energy reaches zero. The rate of ncrease n weght s determned by the parameter c LP, the value of the exponental term at f reman = LP, accordng to The weghtng functon s shown n Fgure ( ) LP + 1 f f reman reman < LP. (4.36) LP ln c λ = LP. (4.37) LP

71 59 MP+1 w reserve LP Percentage of ntal energy reserves remanng Fgure 4-6: Low energy cost weghtng functon The objectve cost of unevenly dstrbuted energy costs, c balance, s gven by the standard devaton of the remanng energy reserves of all nodes. The remanng energy reserves for each node are calculated by subtractng the estmated runtme n the course of the current soluton from the energy reserves before the soluton s executed. As the energy levels approach the same value for all nodes, the energy dstrbuton cost decreases. Balanced energy levels decrease the chances that a sngle node wll deplete ts energy reserves far n advance of the rest of the network. The fnal component of the objectve functon s the maxmum speed penalty. The penalty s the maxmum speed constrant enforced as a soft constrant. As descrbed n Secton 4.1., the maxmum speed constrant states that no soluton may requre a moble

72 60 node to travel faster than ts maxmum possble speed. Ths constrant s formally defned by Equaton 4.9. Enforcement as a soft constrant allows solutons whch volate ths constrant to be consdered vald, albet wth large penaltes assgned to ther ftness. It was found that, n the case of a moble node speed dsadvantage, the use of a hard speed constrant often led to the evolutonary search performng poorly due to the large number of solutons elmnated as nvald. The use of a soft constrant allows nvald solutons to be used as ntermedate solutons untl more ft, vald solutons are found. The addton of a large penalty ensures that solutons n volaton of the constrant are not consdered optmal. The maxmum speed penalty s calculated for each node and s gven by c penalty = N = 1 pen fxed + pen 0 varable ( t t ) follow wndow t t follow follow t > t wndow wndow, (4.38) where the length of the survellance nterval s gven by t follow = t t (4.39) brk nt and the maxmum survellance wndow allowed by the constrant s gven by t wndow rdetect =. (4.40) v v c n The parameter, pen fxed, s a large constant to ensure that even small volatons of the maxmum speed penalty are heavly penalzed. The overall ftness functon combnes the four above objectve components nto a sngle ftness value. Ths ftness value s calculated as the weghted sum of the free tme cost, requred energy cost, energy dstrbuton cost, and maxmum speed penaltes.

73 Solutons wth lower ftness values are more ft, whle solutons wth hgher ftness values are less ft. The fnal ftness functon for a canddate soluton s gven by 61 ftness = w c + w c + w c + w c. (4.41) f freetme e energy b balance p penalty Runtme Calculaton A node s runtme s an estmate of the amount of tme that the node wll spend travelng under ts own power n the course of executng a soluton. Snce the nodes are autonomous, ths value cannot be calculated exactly, snce the exact course the nodes wll follow s not known by the central controller. However, snce the nodes wll be travelng to known postons whch are relatvely far apart, runtme can be estmated to a suffcently accurate degree by assumng a pecewse lnear path. A node whch s nactve n the current soluton has only one task, staton keepng. Ths node must move to ts assgned staton and hold poston there. The response of a node whch s actve s broken nto three segments: ntercept, follow, and return. The ntercept segment s the frst porton of the response of a node to a contact. Durng the ntercept segment, the node moves from ts staton to the poston where t wll ntercept the contact. Ths poston s defned by the contact track and the ntercept tme from the soluton. When calculatng the ntercept runtme, the effect of ocean currents must be taken nto account. For the purposes of all runtme calculatons, a sngle vector for each node, v o, s used to represent the speed and drecton of the ocean current near that node. An example ntercept segment s shown n Fgure 4-7, separated nto powered

74 and drftng sectons. The moble node begns at (x n,y n ) before the soluton s executed and moves to the ntercept poston, (x nt,y nt ) (x nt,y nt ) v tnt o Drft movement Powered movement y [km] 75 B y v runtmentcp n 70 B x (x n,yn ) x [km] Fgure 4-7: Runtme calculaton for ntercept segment The angle of the powered segment,φ, s calculated usng B B x y = x = y nt nt φ = atan x n y ( B, B ) y n v x ox v t oy nt t nt. (4.4) The ntercept component of the runtme s then calculated usng runtme ntercept where v n s the maxmum speed of node. Bx =, (4.43) v cosφ n

75 63 The follow segment s the second porton of the response. It s durng the follow segment that the moble node pursues the target and provdes survellance. In order to follow the target, the node must move under ts own power the entre tme. The follow segment component of the runtme s then defned by runtme follow = t t. (4.44) brk nt The return segment s the fnal porton of the response. Durng the return segment, the node ends target survellance and returns to ts assgned staton. Dependng on the drecton of travel, the current may hnder or assst the movement of the node. If the current can be used to assst the return trp of the node, the requred runtme s the amount of tme requred to move to a poston where the current wll return the node to ts staton, as shown n Fgure 4-8. The node begns at ts break-off pont, (x brk, y brk ), and ends at ts assgned staton, (x s, y s ).

76 64 85 (x s,ys ) 80 v tdrft o y [km] 75 Drft (v +vo)runtmereturn n 70 (x brk,y brk ) Powered d channel x [km] Fgure 4-8: Runtme calculaton for return segment In order to determne f the current can be used to assst the return trp of a node, the drect return headng s frst calculated usng drect ( y y ) ( x x ) φ = atan,. (4.45) s Next, the maxmum devaton between the drecton of the current and the drect headng for whch the current can stll assst s calculated accordng to brk s brk Δ max π = atan v v o n. (4.46) If the condton θ φdrect < Δmax (4.47)

77 s met, where θ s the headng of the current near the th node, then the current wll assst the return trp of node. If ths s the case, then the dstance from the node to the channel whch wll carry the node to ts staton s calculated wth dchannel ( θ + π ) ( ) θ + π The runtme for the return segment s then calculated usng cos = ( x s xbrk ). (4.48) sn 65 runtme return dchannel = (4.49) v n and the return headng (the drecton whch the node wll attempt to travel) s gven by φ return θ π θ + π φ drect = φdrect < θ. (4.50) θ If the current cannot assst the return trp, then the return headng s calculated usng v o φ return = asn sn drect + v n ( φ θ ) φ drect (4.51) and the runtme for the return segment s determned by runtme return v = v 0 n n x cos y sn brk x ( φ ) return brk ys ( φ ) return s + v + v ox oy x brk y brk o. w. xs y. (4.5) s The fnal runtme component s due to staton keepng (holdng poston). The runtme due to staton keepng s an estmate of the amount of tme that a node must spend runnng n order to hold ts poston aganst the current. Frst the staton tme must

78 be calculated. To do ths, the poston where a node begns drftng must be calculated 66 usng x y drft drft = = ( vn cos( φreturn ) + vox ) v sn( φ ) + v ( ) runtme n return oy runtme return return. (4.53) The dstance each node must ntally drft to complete ts return trp and arrve at ts staton s then calculated usng d drft [ x y ] [ x y ] =. (4.54) drft drft s s The staton tme s the amount of tme a node wll spend keepng staton after completng ts return trp. Snce t s not known n advance when a node wll next stop keepng staton, n order to calculate staton tme, a fnte tme horzon s used. The parameter, T horzon, determnes how far beyond the current tme staton tmes are evaluated. Usng ths value, the staton tmes for each node are calculated accordng to t staton = T horzon t + 0 nt ( runtme + t ) runtme return ntercept brk d v drft o f node actve f node nactve. (4.55) The runtme due to staton keepng s estmated usng the rato of the speed of the current to the speed of the node. For example, f the current speed s α m/s and the node speed s α m/s, then the node wll spend α/(α) = 1/ of ts tme correctng ts poston whle keepng staton. Under ths assumpton, the fnal runtme component s gven by runtme staton = t staton v v o n. (4.56)

79 gven by Once all runtme components have been calculated, the total run tme for node s 67 runtme runtmentercept + runtme follow + runtmereturn + = runtme. (4.57) staton These calculatons are performed for each node, each tme a soluton s evaluated. 4. Mxed Integer Optmzaton As formulated n Secton 4.1, the asset allocaton problem forms a nonlnear mxed-nteger optmzaton problem, also referred to as a mxed-nteger nonlnear programmng (MINLP) problem. A mxed-nteger problem s an optmzaton problem n whch the soluton contans both dscrete (nteger) and contnuous parameters. The ncluson of nteger parameters means that many tradtonal optmzaton technques, whch are meant only for functons of contnuous varables, can no longer be appled. In addton, the optmzaton objectve s a nonlnear functon of the dscrete and contnuous parameters. Ths further lmts the number of applcable optmzaton technques. Despte the numerous dffcultes, much attenton has been focused on the soluton of MINLP problems. Ths s due n large part to the frequency wth whch ths type of problem arses n many felds. Examples of nonlnear mxed nteger optmzaton problems can be found n mechancal system desgn [39] and ntegrated crcut desgn, ncludng multplexer desgn [40] and System on Chp (SoC) desgn [41]. Numerous examples n the power sector nclude: power plant desgn [4], power generaton expanson plannng [43], and power transmsson expanson plannng [44]. Examples n the communcaton feld nclude energy management [45] and access pont placement

80 68 [46] n wreless networks. Industral examples appear n tasks such as dstrbuton chan plannng [47] and job shop schedulng [48]. Snce no general soluton to a MINLP problem s known, one of two approaches s generally taken. The frst approach s to somehow reduce the complexty of the problem so that a determnstc mathematcal programmng approach can be appled. Typcal examples of ths nclude Branch and Bound [44, 46], Generalzed Benders Decomposton [43], and Outer Approxmaton [49]. There are many varatons of these approaches, but each generally requres the problem meet strct condtons or be subject to certan smplfcatons. Some requre condtons on the objectve functon such as convexty, dfferentablty, or separablty n dscrete and contnuous varables. Some approaches requre that the problem s dvsble nto an nteger master problem and contnuous sub-problems [43]. The second approach s to use a heurstc optmzaton or stochastc search technque. These technques make few or no requrements on the optmzaton functon or problem structure. Ths means that they can be appled to nearly any MINLP problem. Because they are not exhaustve, stochastc methods scale well wth problem sze. However, they are also not guaranteed to converge to the global optmum. The lack of strct requrements on problem structure for stochastc technques has made them very popular for nonlnear mxed-nteger problems. In partcular, evolutonary algorthms have become a common tool for solvng MINLP problems. Examples of evolutonary algorthm applcaton to MINLP problems ncludes Genetc Algorthms [4], Dfferental Evoluton [39], and Partcle Swarm Optmzaton [47]. Whle not guaranteed to generate the globally optmal soluton, these approaches have been shown n many cases to

81 69 generate near-optmal solutons to complex problems for whch determnstc technques cannot be appled. The asset allocaton problem s poorly suted to a determnstc technque. Its objectve functon s very complex, non-dfferentable, non-convex, and nonlnear. Its soluton space s very large as well. For a problem wth N moble nodes, there are ( N )N! combnatons of dscrete parameter values and N contnuous parameters. In addton, the ftness of a set of contnuous parameters n a soluton s tghtly coupled to the dscrete parameters n the soluton. Ths means that an optmal set of contnuous soluton parameters for a gven dscrete parameter combnaton may be far from optmal for a dfferent set of dscrete parameters. For these reasons, a stochastc search technque was chosen as the bass of the soluton approach to the asset allocaton problem. 4.3 Evolutonary Strateges An evolutonary algorthm (EA) was chosen as the bass of the asset allocaton soluton approach due to a number of desrable propertes. Frst, because they are not exhaustve, EAs typcally scale well to large problems wthout dramatc ncreases n runtme. In addton, EAs can work wth arbtrarly complex objectve functons, wthout placng restrctons on the type of functon. Fnally, as many authors have shown n recent decades, EAs can produce optmal or near-optmal solutons to many problems, despte beng stochastc technques. Due to a number of benefts, Evolutonary Strateges (ES) was chosen as the EA used as the bass of the asset allocaton optmzaton strategy. Evolutonary Strateges

82 70 use of a natural soluton space, rather than an encoded soluton space, allowed for the easy ntegraton of problem specfc operators. In addton, ES supports mxed nteger problems wth lttle trouble. In some EAs, such as many Dfferental Evoluton approaches, nteger parameters are temporarly treated as contnuous values then rounded to the nearest nteger value [39, 50]. In the asset allocaton problem, where fractonal values for dscrete parameters have no meanng and dscrete parameter values whch are near each other sequentally may have vastly dfferent physcal meanngs and ftness values, such an approach would not be benefcal. Evolutonary Strateges was frst envsoned n 1963 by Rechenberg and Schwefel [5]. ES shares many smlartes wth Evolutonary Programmng (EP) and as a result, many see them as varatons of the same algorthm. A thorough dscusson of the development of ES and dfferences between ES and EP can be found n [5]. Snce ts creaton, ES has found wdespread use n many felds, generatng near-optmal solutons to many problems that were otherwse consdered ntractable. Examples nclude applcatons n vdeo compresson [51, 53], power systems [54, 55], computer vson [56], cryptography [57], and neural network tranng [58, 59]. Lke other evolutonary algorthms, ES mantans a populaton of potental solutons. Each generaton, ths populaton s manpulated by recombnaton and mutaton operators, generatng new solutons. Solutons are then evaluated for ftness and the selecton process determnes whch wll survve to the next generaton. Ths process s repeated untl a stoppng crteron s reached. The evolutonary search algorthm presented n ths chapter s based on the Evolutonary Strateges algorthm wth numerous modfcatons made to mprove performance for the asset allocaton problem. A flowchart

83 showng the operaton of the ES algorthm for the asset allocaton problem s shown n Fgure Start Intalze populaton gen = 0 gen = gen + 1 Update global factor Update parameters Recombnaton Mutaton operators Mutaton Ftness evaluaton Selecton No Termnaton crtera met? Yes End Fgure 4-9: Flowchart for asset allocaton Evolutonary Strateges algorthm The algorthm begns by creatng a randomly ntalzed populaton of solutons. These solutons form the parent populaton, wth μ solutons, n the frst generaton. The process of creatng the ntal populaton s dscussed n Secton At the begnnng of each generaton, the current global factor value s updated. Ths value controls search

84 7 scope and s used to update the values of most algorthm parameters. The reason for the global factor, the manner n whch t s updated, and the parameters whch t controls are dscussed n Secton After the parameters are updated, recombnaton s used to form a chld populaton wth λ solutons. Each soluton n the chld populaton s formed from one or more solutons n the parent populaton usng recombnaton. The process of recombnaton combnes parameters from multple solutons to form a sngle soluton and s dscussed n Secton Followng recombnaton, problem specfc mutaton operators are appled to the chld populaton. These mutaton operators manpulate the parameters of the chld solutons n an attempt to mprove ther ftness value. Mutaton operators are randomly appled to each chld accordng to probabltes defned by algorthm parameters. The mutaton operators used n ths algorthm are presented n Secton After the mutaton operators have been appled, drect parameter mutatons are appled at a rate defned by algorthm parameters. These standard mutatons manpulate ndvdual parameters and are dscussed n Secton After each chld soluton s mutated, t s checked to ensure that t satsfes the constrants of Secton 4.1. and the boundares of Secton Any mutaton whch causes the boundares or constrants to be volated s rejected and re-performed. After the chld populaton has been formed and mutated, each soluton s evaluated for ftness. Solutons whch have a lower ftness value are consdered better (more ft). The objectve functon used n the ftness evaluaton s gven n Secton Once all chldren have been evaluated, the selecton process forms a new populaton from the parent and chld populatons of the current generaton. Ths new populaton wll be

85 73 the parent populaton of the next generaton. The methods and crtera used n the selecton process are dscussed n Secton Ths completes one generaton of the algorthm. At the end of each generaton, the termnaton crtera dscussed n Secton are checked. If these condtons are met, the search ends and the best soluton s chosen. If the condtons have not been met, the search contnues for another generaton. 4.4 Applcaton of Evolutonary Strateges to Asset Allocaton In order to apply ES to the asset allocaton problem, t must be determned how each of the components of ES wll be appled to the problem. Ths frst requres determnng the form n whch canddate solutons wll be represented. The manner n whch an ntal populaton of these solutons s generated must then be decded. Next, the recombnaton, selecton, and mutaton operators must be formulated and adapted to ths problem. Fnally, t must be determned how search scope s controlled, and what search termnaton condtons wll be used. These ssues wll be dscussed n the followng sectons ES Soluton Format Snce the approach taken here s ES-based, solutons are represented n ther natural soluton space. The natural soluton space for the asset allocaton problem s presented n Secton The ES soluton uses the same parameters as those lsted n Table 4-1, wth one excepton. Whle an asset allocaton soluton only requres ntercept

86 74 and break-off tme parameters for nodes whch are actve, an ES soluton stores these parameters for all nodes. Ths s because the ES solutons are partal solutons, used n the process of an evolutonary search. A node whch s not actve n a gven soluton may have ts status changed to actve n the next generaton. Therefore, t s mportant that ntercept and break-off tme parameters are retaned, even when the node s currently set to nactve status. Once a fnal soluton s chosen, the ntercept and break-off tme parameters for nactve nodes are dscarded. Wth the slght modfcaton above, the ES solutons are gven by a collecton of parameters n ther natural representaton. Such solutons are of the form where the soluton components are gven by actve t t nt brk staton { actve, t, t staton } soluton =,, (4.58) = ( k ) ( k ) nt ( ) brk ( ) ( k ) ( k ) [ 1 N 1 N ] ( k ) = N 1 ( ) [ ] k nt nt nt nt ( k ) = t t N 1 ( ) [ ] k brk brk brk brk ( k ) = actve t 1 1 t staton actve L L t t staton L ( k ) [ 1 N 1 N ] ( k ) wth k equal to the ndex of the soluton n the current generaton and N equal to the number of avalable assets consdered n the problem. t k N t L N k actve staton actve staton, (4.59) 4.4. Populaton Intalzaton When the ES search s begun, a parent populaton must be created. Ths means choosng μ ntal solutons, where μ s the number of ndvduals, or canddate solutons, n the parent populaton. In order to help avod convergence to a suboptmal soluton, the

87 75 ntal populaton should be well dstrbuted n the soluton space. In order to facltate ths, all ntal soluton parameters except for staton assgnments are chosen at random. These parameters are chosen randomly usng the boundares establshed n Secton A soluton whch volates the orderng constrant s regenerated. By defnng the operaton UnfRand {A} as the act of unformly choosng a random element of set A, the parameters of soluton (k) are determned accordng to actve = UnfRand t nt t brk = UnfRand{ B } T = UnfRand{ B } { 0,1} T 1 N. (4.60) Unlke the other parameters, the staton assgnments are not randomly assgned. Snce drastc changes n staton assgnments are not lkely to occur n the optmal assgnment 3, staton assgnments n the ntal solutons are not randomly ntalzed. Instead, they are set to the current staton assgnments of the network, leavng mutatons responsble for explorng dfferent staton mappngs Recombnaton The recombnaton process s used to create a chld soluton whch nherts the propertes of one or more randomly selected parent solutons. Ths process s repeated multple tmes n order to form a populaton of chld solutons. Whle some chldren wll be exact duplcates of a parent soluton, others wll nhert trats from multple parents. In 3 Drastc changes n staton assgnments would lkely cause large energy costs wth no mprovement n free tme to offset them.

88 76 some cases, favorable trats may be nherted from multple parents, resultng n a chld soluton wth a lower ftness cost than any of ts parents. The propertes nherted by the chld soluton take the form of the soluton parameters of the parents. An example of a chld soluton formed through the recombnaton of N p randomly selected parent solutons s shown n Table 4-3. The parent parameters hghlghted n red are those whch contrbute to the chld soluton. The actve flag for node n the chld soluton s selected at random from the actve flags of node for each of the parent solutons. Ths s repeated for each of the N nodes n the soluton. Unlke the actve flags, staton assgnments are not nherted from multple parents. Ths s done to prevent overlappng staton assgnments from appearng n the chldren. As fewer parents are used, the number of combnatons whch result n unque chld staton assgnments generally decreases. Wth two parents, the only combnaton whch results n unque assgnments n the chld s when all staton assgnments are taken from only one of the parents. For ths reason, all chld staton assgnments are taken from a sngle parent, chosen at random from the N p parents. The ntercept and break-off tme parameters of the chld soluton are assgned usng a weghted average of the parent tme parameters. Ths places the chld soluton tme parameters wthn a convex polytopc regon of the tme parameter soluton space, bounded by the parameters of the parent solutons. The weghts used to combne the parameters, λ k, are unformly chosen at random, such that 0 λ k 1. These weghts are then normalzed such that N p k = 1 λ = 1. (4.61) k

89 The ntercept and break-off tmes for the chld soluton are then calculated usng 77 N p [ tnt ] = k [ tnt ] chld λ k = 1 N p [ tbrk ] = k [ tbrk ] chld λ k = 1 parent( k ) parent( k ) { 1, L, N}. (4.6) Table 4-3: Recombnaton of multple parent solutons to form chld Parameter Actve flag Intercept tme Break-off tme Staton Node 1 N 1 N 1 N 1 N Parent t 1 nt t nt t N nt t 1 brk t brk t N brk N 1 Parent t 1 nt t nt t N nt t 1 brk t brk t N brk 1 N Parent N p t 1 nt t nt t N nt t 1 brk t brk t N brk 1 N Chld t 1 nt t nt t N nt t 1 brk t brk t N brk 1 N The number of parents whch wll be used n the recombnaton operaton, N p, s randomly determned, accordng to UnfRand ln N p = cel, (4.63) precom ln where UnfRand s a unformly dstrbuted random number between 0 and 1. The recombnaton rate parameter, p recom, controls the selecton of the number of parents to be used n the recombnaton process. The value of p recom s the probablty that more than one parent wll be used. In the case where N p = 1, a chld s an exact copy of ts parent. Although the chances are very low, when usng the above formula t s possble for N p to be very large. To prevent ths, N p s allowed to be no larger than 5. The probablty mass

90 functon for the number of recombnaton parents, P[N p ], s shown n Fgure The values of ths PMF are gven by 78 P [ N ] p ( 1 p ) p = 4 precom 8 0 recom N p 1 recom 1 p recom N N N p p p = 1 {,3,4} = 5 otherwse. (4.64) PMF of recombnaton parent quantty P[N p ] Fgure 4-10: Recombnaton parent PMF N p Fgures 4-11 and 4-1 present an example of the recombnaton process. Fgure 4-11 shows two parent solutons, whch wll be combned to form a chld soluton. Each soluton s represented wth approxmate pecewse lnear trajectores for each node, based on current soluton parameters. Fgure 4-1 shows the resultng chld soluton produced by the recombnaton operaton. As can be seen, all ntercept and break-off

91 tmes n the chld soluton are bounded by those of the parents. In addton, not all nodes are actve n the chld soluton A Moble node Contact track Approxmate trajectory: parent 1 Approxmate trajectory: parent y [km] B 30 D C x [km] Fgure 4-11: Example recombnaton operaton parents before recombnaton

92 A Moble node Contact track Approxmate Approxmate trajectory: trajectory: Parent solutons Chld soluton y [km] B 30 D C x [km] Fgure 4-1: Example recombnaton operaton chld after recombnaton Mutaton After a populaton of chldren s created through recombnaton, these chldren are subjected to random mutatons. These mutatons take the form of random changes to soluton parameters. The mutatons can make a small change to a soluton or drastcally change ts locaton n the search space. Mutaton s the prmary means through whch ES explores the search space. The probablty of any gven parameter mutatng s constant across all nodes and all solutons n the chld populaton. Gven ths fact, the followng mutaton operatons are performed for all solutons n the chld populaton.

93 81 The frst parameters to be mutated for each soluton are the actve flag parameters. Snce each actve flag can only take on two values, a mutaton causes the parameter to swtch values. These mutatons take place for each soluton accordng to actve actve 1 UnfRand pmutactve = 1 N, (4.65) actve UnfRand > pmutactve where s the XOR operator and UnfRand s a unformly generated random number between zero and one whch takes a new random value for every node n every soluton. The parameter pmut actve gves the mutaton probablty of any gven actve flag parameter. The next parameters to mutate are the ntercept and break-off tmes. Although these parameters mutate ndependently, the manner n whch they mutate s dentcal. Each of the tme parameters for each node n each soluton mutates accordng to a mutaton probablty parameter, pmut tme. A tme parameter s mutated by the addton of a zero-mean, normally dstrbuted random number. The degree of ths mutaton s controlled by the parameter σ tme. Ths parameter controls the standard devaton of the mutatons, expressed as a fracton of the allowable range of tme values for each node. Therefore, the tme mutatons take place accordng to t nt t = nt + NormRand ( σ )[ max( B ) mn( B )] tme tnt T T UnfRand UnfRand > pmut pmut tme tme (4.66) and t brk t = brk + NormRand ( σ )[ max( B ) mn( B )] tme tbrk T T UnfRand UnfRand > pmut pmut tme tme (4.67)

94 8 for all 1 N for all chld solutons. Agan, UnfRand s a unformly dstrbuted random number between zero and 1 whle NormRand s a normally dstrbuted random number wth a mean of zero and a varance of one. Snce Equatons 4.66 and 4.67 do not enforce any boundares or constrants, t s possble that the ntercept and break-off tmes could be mutated such that they volate the boundary condtons or the orderng constrant. To prevent ths, mutated chld solutons are checked to confrm that all boundares and constrants are satsfed. If a mutated soluton s found to volate the boundares or constrants, the mutatons are rejected and new mutatons are appled. Due to the staton unqueness constrant, drect mutaton of staton assgnments s not possble. The reassgnment of a sngle node to a new staton wll result n overlappng assgnments snce another node wll have already been assgned to that staton. In order to allow for the reassgnment of statons, a staton swap operator s used. The staton swap operator exchanges the staton assgnments of two nodes, thus allowng staton assgnments to be mutated wthout volatng the unqueness constrant. The probablty of a staton swap mutaton beng appled to a chld soluton s gven by pmut staton. When the operaton s appled, two nodes are unformly selected at random from the range 1,j N. The values of staton and staton j are then exchanged. No more than one staton swap s appled to a sngle soluton n a gven generaton Mutaton Operators Although the basc mutaton operatons descrbed n Secton allow for exploraton of the soluton space, ths exploraton s random and undrected. Random

95 83 exploraton s an mportant aspect of an evolutonary search, helpng the search to avod local mnmums. However, due to the nature of the asset allocaton problem, random mutatons alone were found to be nsuffcent to generate satsfactory results. The shortcomngs of the basc mutaton operatons of the prevous secton are llustrated n the followng example. One major ssue wth basc mutatons s ther nablty to effectvely address confguraton change. The confguraton s the set of all actve flags n a gven soluton. The specfcaton of whch nodes wll or wll not be nvolved n contact survellance for a gven soluton s referred to as the confguraton of that soluton. Whle the basc mutaton operatons allow for exploraton of confguratons through the mutaton of actve flags, these basc mutatons may not always lead to mproved solutons. A smple example of such a stuaton s shown n Fgure Ths fgure shows a porton, ncludng moble nodes, of a potental asset allocaton soluton. In the vsualzaton of ths soluton, astersks represent the postons specfed by the ntercept and break-off tmes n the soluton. Approxmate trajectores for each node are marked wth lnes. These lnes mark a path from the startng poston for each node to ts ntercept poston, then to ts break-off poston, and fnally to ts assgned staton. Sold lnes ndcate a node whch s actve n the gven soluton, whle dotted lnes ndcate an nactve node. All fgures of ths type wll use the same formattng, so the legend wll not be repeated.

96 Moble node Approxmate trajectory (actve) Approxmate trajectory (nactve) Contact track Intercept/break-off poston y [km] A 40 B x [km] Fgure 4-13: Local mnmum due to confguraton change ntal soluton In the ntal soluton shown n Fgure 4-13, only node A s actve. Actvatng node B and splttng the coverage nterval 4 between the two nodes would result n mproved soluton ftness. However, n order to accomplsh ths, the actve flag of node B must be set and the break-off tme of node A must be changed as shown n Fgures 4-14(a) and 4-14(b), respectvely. Unfortunately, ether of these ntermedate solutons are less ft than the ntal soluton. The soluton shown n Fgure 4-14(a) has a hgher ftness cost because the actvaton of node B ncreases the overall energy consumpton wthout decreasng contact free tme. The soluton of Fgure 4-14(b) has a hgher ftness cost because t greatly ncreases contact free tme. 4 The coverage nterval s the porton of the contact track assgned to a node for survellance, defned by the ntercept and break-off tmes.

97 y [km] 50 A y [km] 50 A 40 B 40 B x [km] (a) x [km] Fgure 4-14: Local mnmum due to confguraton change ntermedate solutons: node B set to actve (a), node A break-off tme altered (b) (b) A soluton wth mproved ftness over the ntal soluton s shown n Fgure Although the ntal soluton of Fgure 4-13 could eventually reach ths form through basc mutatons, many ntermedate solutons would be requred, such as those n Fgure Snce most of these ntermedate solutons wll have hgher ftness costs, they are less lkely to survve between generatons.

98 y [km] A 40 B x [km] Fgure 4-15: Local mnmum due to confguraton change mproved soluton Although n ths example, the ntal soluton could potentally be mutated nto an mproved soluton n small number of generatons, more complex examples exst. There are many other local mnmum stuatons where an mprovement n ftness would requre basc mutatons over many generatons. The more generatons of hgh ftness cost ntermedate solutons requred, the less lkely t s that an mproved soluton wll be reached. In order to attempt to escape these local mnma n many common stuatons, problem-specfc mutaton operators are used. The mutaton operators are appled to the chld populaton after the recombnaton process. A parameter for each operator specfes the probablty that t wll be appled to each chld. The followng sectons dscuss the seven mutaton operators whch are used by the evolutonary search n addton to the basc mutaton operatons. These mutaton

99 operators do necessarly not move the search to an optmal locaton, but attempt to move t closer Splt Operaton The purpose of the splt operaton s to escape the local mnmum llustrated n the prevous secton. Ths local mnmum can occur when a sngle node s assgned a large coverage nterval. In such stuatons, ftness can often be mproved by dvdng the coverage nterval between multple nodes. Ths reduces the maxmum energy cost burden to any sngle node. Ths may result n a lower ftness score by balancng energy consumpton between multple nodes. It wll, however, ncrease overall energy consumpton. Ths s because a two node response results n a greater total node travel dstance than a one node response, except under very lmted crcumstances. Ths fact s llustrated by Fgure 4-16.

100 b 1 70 B b 60 b 3 y [km] 50 a 4 40 a 1 30 a A 0 a x [km] Fgure 4-16: Moble node travel dstance for one and two node responses In the Fgure 4-16, the sold lnes mark the trajectory of nodes A and B when both respond to the contact. The dashed lnes mark the trajectory of node A when t responds alone. In both cases the total coverage s the same. When node A responds alone, the total dstance traveled by all nodes for the sngle node response s gven by ( a + b + a + ) ( 0) d sngle = d A + d B = 4 a 3 +. (4.68) When node A and B respond together, the total dstance traveled by all nodes for the double node response s gven by d ( a + a + a ) + ( b + b ) double d A + d B = b3 =. (4.69) Subtractng these dstances gves

101 d double d sngle = a1 + ( b1 + b3 ) ( a + b ) a a 4 1 a 4 = 0 4 a 4. (4.70) 89 It can be seen from Equaton 4.70 that the total travel dstance for a double node response s always greater than or equal to that of a sngle node response. In addton, the only tme the two dstances are equal s when b 1 +b 3 = b and a 1 +b = a 4. These condtons are only met when both nodes A and B le on the contact track. Therefore, n all but the degenerate case, the total dstance s always less for the sngle node response. Due to ths fact, the splt operaton can only lead to mproved ftness values f t mproves the balance of energy costs between nodes. When appled to a soluton, the splt operaton begns by randomly choosng two dfferent nodes. Node A must be actve n the current soluton whle node B may be actve or nactve. Next, a handoff tme s randomly chosen. After the coverage nterval s splt, the handoff tme s the tme at whch one node wll break-off survellance and the other wll begn. The handoff tme s unformly chosen wthn the coverage nterval, accordng to handoff A A A ( t t ) t t = UnfRand +. (4.71) The node whch s closest to the current contact poston s gven the frst segment of the dvded coverage nterval. Based on ths orderng, the coverage nterval s dvded by settng the ntercept and break-off tmes accordng to brk nt nt

102 90 t A nt t A brk t B nt t B brk = t = t A nt = t = t handoff handoff A brk (4.7) when node A s assgned the frst segment and t A nt t A brk t B nt t B brk = t = t = t handoff A brk A nt = t handoff (4.73) when node B s assgned the frst segment. Fnally, the actve flag of node B s set, completng the operaton. An example soluton, before and after a splt operaton, s shown n Fgure B 70 B y [km] 50 y [km] 50 t handoff A 30 A x [km] x [km] (a) (b) Fgure 4-17: Splt operaton example before operaton (a) and after (b)

103 Combne Operaton 91 The combne operaton performs the opposte functon of the splt operaton. The purpose of the combne operaton s to merge the coverage ntervals of two nodes nto that of one. In some cases, ths wll result n an mproved ftness score. Examples of ths nclude a node whch s low on energy or a node whch must travel a great dstance to reach the contact. Fgure 4-18 shows an example of a soluton whch may beneft from the combne operaton. In ths example, ftness would be mproved f node A were to be set to nactve and ts coverage nterval added to that of node B. To perform these changes usng basc mutatons, node A would have to be set to nactve and the break-off tme of node B would have to be greatly ncreased. Ether of these mutatons on ther own would ncrease the ftness cost. In order to ad the escape from ths potental local mnmum, the combne operaton performs these operatons as one.

104 B 60 A y [km] Fgure 4-18: Combne operaton motvaton x [km] When the combne operaton s performed on a soluton, t begns by randomly selectng two actve sequental nodes. If less than two nodes are actve, the operaton s not performed. In order to dentfy whch nodes are sequental, the node ndces are sorted accordng to ntercept and break-off tme. The sorted ndces are contaned n the vectors w and v, where t w1 nt t v1 brk t t w nt v brk L t w nt L t N 1 v brk N 1 t t wn nt vn brk. (4.74) To begn, node A s randomly selected from the actve nodes n the current soluton such that at least one of the followng condtons s met: w N A v A 1 (4.75)

105 93 If both of these condtons are met, then the coverage nterval of node A s enclosed by those of all other nodes and a new A must be chosen. Once node A s chosen, node B s chosen at random from the remanng actve nodes and tested to see f t s sequental wth node A. The set {A,B} s sequental f the ntercept of node B s the frst after the ntercept of node A,.e. w = A. (4.76) w = B +1 The set {B,A} s sequental f the break-off of node A s the frst after the break-off of node B,.e. v = A. (4.77) v = B 1 If nether of these condtons s met, a new node B s chosen at random untl one of the condtons s met. Once node B s chosen, node A s set to nactve status and the ntercept and break-off tmes of node B are updated accordng to t t B nt B brk = mn = max A B { tnt, tnt} A B { t, t } An example soluton, before and after applcaton of the combne operaton, s shown n Fgure brk brk. (4.78)

106 B 70 B 60 A 60 A y [km] 50 y [km] x [km] x [km] (a) (b) Fgure 4-19: Example combne operaton before operaton (a) and after (b) Swtch Operaton The swtch operaton reassgns the coverage nterval of one node to another node. Ths operaton can result n mproved ftness when a node s assgned a coverage nterval whch could be assgned to another node for a lower energy cost. Examples of ths nclude replacng one node wth another whch has hgher energy reserves or one whch s closer to the coverage nterval. When the swtch operaton s appled to a soluton, two nodes are selected at random. Node A must be actve n the current soluton whle node B may be actve or nactve. Once the nodes are selected, node A s set to nactve and node B s set to actve

107 95 status. The ntercept and break-off tmes of node B are then set equal to those of node A. An example of the applcaton of the swtch operaton s shown n Fgure 4-0. In ths example, ftness s mproved snce energy costs are reduced whle mantanng the same coverage B 70 B y [km] 50 y [km] A 40 A x [km] x [km] (a) (b) Fgure 4-0: Swtch operaton example before operaton (a) and after (b) Exchange Operaton The exchange operaton behaves smlarly to the swtch operaton wth the excepton that the transfer of parameters s two-drectonal rather than one. In the exchange operaton, the ntercept and break-off tmes of one node are swapped wth those of another. Ths operaton s of partcular use n stuatons such as the one shown n

108 96 Fgure 4-1(a). Ths soluton s at a local mnmum because any small change would result n an ncrease n ftness cost. Reachng the mproved soluton n Fgure 4-1(b) requres all four tme parameters to be changed by a large amount. Snce ths s extremely unlkely to occur n a sngle generaton usng basc mutatons, multple ntermedate solutons would be requred. Durng these ntermedate generatons, the coverage ntervals would need to cross over one another. In these ntermedate solutons, such as the one shown n Fgure 4-, the overlappng coverage results n greater contact free tme whch ncreases ftness cost. Therefore, t s unlkely that all the ntermedate solutons necessary to reach the soluton of Fgure 4-1 (b) would survve. The exchange operaton performs ths task n a sngle generaton, as shown n Fgure B 70 B y [km] 50 y [km] A 30 A x [km] (a) x [km] Fgure 4-1: Exchange operaton example before operaton (a) and after (b) (b)

109 B 60 y [km] A x [km] Fgure 4-: Less ft ntermedate soluton avoded by exchange operaton When the exchange operaton s appled to a soluton, t begns by choosng two nodes at random. These nodes can be actve or nactve n the current soluton. The two nodes then swap ntercept tmes and swap break-off tmes. The actve status of both nodes remans unaffected by ths operaton Slde Operaton The purpose of the slde operaton s to make the process of movng a handoff locaton smpler. A handoff locaton s the locaton where one node breaks off from the contact and another node ntercepts the contact. An example of a handoff locaton s shown n Fgure 4-3. Through smple mutatons, ths handoff locaton s modfed by alterng t A brk and t B A nt. In these terms, the handoff locaton n Fgure 4-3 s at δt brk = 0,

110 98 δt B A nt = 0. The ftness score for the regon near ths soluton, obtaned by varyng δt brk and δt B nt, s shown n Fgure 4-4. The same plot s shown n Fgure 4-5 wth a smaller scale. As these plots ndcate, the handoff pont n Fgure 4-3 s not at a local mnmum. However, the ftness ncreases quckly f t A brk t B nt. Ths s because when these tmes are not equal there s ether an ncrease n free tme or an ncrease n energy cost wthout a decrease n free tme. Ether of these cases results n an ncrease n ftness cost. In order to easly allow for exploraton of the regon where t A brk = t B nt, the slde operaton s used A 60 y [km] B x [km] Fgure 4-3: Slde operaton motvaton suboptmal handoff locaton for two nodes

111 99 Fgure 4-4: Slde operaton motvaton ftness score n regon of ntal soluton δt brk a = 0 δt nt b = 0 Fgure 4-5: Slde operaton motvaton ftness score n regon of ntal soluton (small scale) When the slde operaton s appled to a soluton, t frst ensures that at least two nodes are actve n the current soluton. If less than two are actve, the operaton cannot

112 100 be appled. It then selects a node, k, at random from the actve nodes. The nodes whose coverage ntervals occur before that of node k are then determned,.e. nodes whch satsfy t t. (4.79) brk k nt Nodes wth coverage ntervals after node k are also determned, satsfyng t t. (4.80) nt k brk If no nodes satsfy ether of these condtons, the operaton s abandoned. If no nodes satsfy condton 4.79, node m s chosen as the node whch satsfes condton 4.80 wth the mnmum ntercept tme,.e. the frst node after node k. If no nodes satsfy condton 4.80, node m s chosen as the node whch satsfes condton 4.79 wth the maxmum break-off tme,.e. the frst node before node k. If at least one node satsfes both condtons, node m s randomly chosen as the frst node before or after node k. Gven nodes k and m, the node wth the frst coverage nterval s referred to as node B and the second s referred to as node A. The drecton of the slde s then randomly chosen, ether forward or backward n tme. The slde tme, δt, s then chosen at random from zero up to a fracton of the length of the coverage nterval n the drecton of the slde. For forward sldes ths s the length of the coverage nterval of node A, for backward sldes ths s the length of the coverage nterval of node B. The parameter SldeFact s an algorthm parameter and controls the maxmum magntude of the slde length as a fracton of the coverage ntervals. Usng ths defnton, the slde tme s defned by

113 B B ( SldeFact)( tbrk tnt ) A A ( SldeFact)( t t ) UnfRand UnfRand 0.5 δ t =. (4.81) UnfRand brk nt UnfRand > 0.5 Once the slde tme s calculated, the handoff pont s updated usng t B brk t A nt = t = t B brk A nt δt. (4.8) + δt An example llustratng the applcaton of the slde operaton s shown n Fgure 4-6. In ths example, soluton ftness s mproved by decreasng overall energy consumpton and mprovng energy consumpton balance whle mantanng the same overall coverage B 70 B y [km] 50 y [km] 50 δt A 30 A x [km] (a) x [km] Fgure 4-6: Slde operaton example before operaton (a) and after (b) (b)

114 Shft Operaton 10 The shft operaton s used to shft a coverage nterval forward or backward n tme, wthout changng the length of that nterval. Its prmary use s to ad the evolutonary search n the case where some nodes are slower than the contact. In ths case, each node wth a speed dsadvantage has a maxmum bound on the length of ts coverage nterval. Gven a node whch s already assgned ts maxmum length coverage nterval, as shown n Fgure 4-7, mutatons of sngle parameters often result n ncreases n ftness cost. A sngle parameter mutaton wll ether ncrease the length of the coverage nterval, nvokng penaltes, or decrease the length of the coverage nterval, ncreasng free tme. Both of these changes wll result n an ncreased ftness cost. When the shft operaton s performed on a soluton, a node s randomly selected. The actve status of the node s not consdered or altered by the shft operaton. The drecton of the shft, forward or backward n tme, s then randomly selected. The shft drecton determnes the sgn of the shft tme, δt. The magntude of the shft tme s a random value, bounded by the length of the coverage nterval of the node. The parameter ShftFact controls the maxmum magntude of the shft as a fracton of the nterval length. Usng ths parameter, the shft tme s calculated as A A ( ShftFact)( tbrk tnt ) A A ( ShftFact)( t t ) UnfRand UnfRand 0.5 δ t =. (4.83) UnfRand brk nt UnfRand > 0.5 Once the shft tme s calculated, the ntercept and break-off tmes are updated usng t t A nt A brk = t = t A nt A brk + δt. (4.84) + δt

115 103 Fgure 4-7 shows an example of the applcaton of the shft operaton. After the operaton s complete, the ftness of the soluton has mproved due to the reducton of travel dstance whle mantanng the same coverage A A δt y [km] 50 y [km] 50 δt x [km] (a) x [km] Fgure 4-7: Shft operaton example before operaton (a) and after (b) (b) Algn Operaton The algn operaton was desgned, not for a specfc local mnmum, but as a general heurstc operator. The purpose of the algn operaton s to brng a soluton whch s far from the optmum nto a standard form, from whch the optmum may possbly be more easly reached. Ths standard form removes coverage gaps and places handoff

116 104 ponts mdway between pars of adjacent nodes. All nodes whch are currently actve reman actve n the mutated soluton. The ntercept tme of the frst node and the breakoff tme of the last node are not altered. Ths standard form dvdes the coverage between all actve nodes and assgns each node the nearest coverage nterval. Once the soluton s n ths form, the applcaton of other mutaton operatons and basc mutatons may be able to move t closer to the optmal soluton, although ths s by no means guaranteed. The algn operaton begns by calculatng the closest pont of approach (CPA) for each actve node n the soluton. The CPA for each node s the pont along the contact track closest to the poston of that node. The dstance from the ntal contact poston to the CPA for each node s calculated as follows d CPA cosψ = ( x xc ) { 1L N} sn, (4.85) ψ where x s the poston of the th node, x c s the ntal poston of the contact, and ψ s the contact headng. The tme value assocated wth each CPA s calculated usng t CPA dcpa = { 1L N}, (4.86) v c where v c s the contact velocty. The CPA tmes are then sorted n ncreasng order. The vector k contans the sorted node ndces, defned by k t k1 CPA { 1L N} { 1L N} t k CPA L t The CPA tmes are then used to calculate handoff tmes, mdway between each CPA pont, usng k N 1 CPA t k N CPA. (4.87)

117 t hand t = k CPA + t k + 1 CPA { 1L( N 1) } 105. (4.88) Usng the handoff tmes, the ntercept and break-off tmes of the actve nodes are updated accordng to t t k brk k + 1 nt = t = t hand hand { 1L( N 1) }. (4.89) Fnally, the ntercept and break-off tmes of the frst and last actve nodes are checked to ensure that the orderng constrant remans satsfed. An example applcaton of the algn operaton s shown n Fgure 4-8. The operaton removes coverage gaps and overlap, partally balances energy costs, and presents a soluton wth mproved ftness from whch to contnue the search A 80 A 70 B 70 B y [km] 50 y [km] D C 40 D C E 0 E x [km] (a) x [km] Fgure 4-8: Algn operaton example before operaton (a) and after (b) (b)

118 4.4.6 Selecton 106 After the recombnaton and mutaton processes are complete, the result s a chld populaton of canddate solutons. Once the chld solutons are evaluated for ftness, the selecton process determnes whch solutons survve to the next generaton. The selecton process chooses the μ solutons whch wll form the next generaton from the combned populaton of μ parents and λ chldren of the current generaton. Ths type of selecton s referred to as (μ+λ)es [5]. Because the parent populaton s ncluded n the selecton process, t s possble for an exceptonally ft soluton to survve ndefntely. The selecton process whch chooses the new populaton s based on two crtera: ensure the survval of ft solutons and mantan populaton dversty. By ensurng the survval of ft solutons, the selecton process guarantees that the result produced by the ES algorthm s the most ft soluton found n the course of the search. The search ensures the survval of ft solutons through the use of (μ+λ)es and eltst selecton. Even f the chld populaton offers no mprovement over the current parent populaton, eltst selecton and (μ+λ)es ensure that the most ft solutons generated up to the current generaton wll be preserved. Addtonally, stochastc tournament, n whch the most ft soluton n a random subset of the populaton s selected wth a gven probablty, meets ths crteron to a lmted extent. Whle t only consders selected subsets of the soluton populatons, stochastc tournament does favor the selecton of ft solutons. The second crteron requres that the selecton process mantan populaton dversty. A dverse populaton of solutons s mportant to avod premature convergence

119 107 of the search to a suboptmal soluton. Low dversty wll often cause the search to settle on a local mnmum, whch may or may not be the global mnmum. Whle ths s the case for most evolutonary searches, t s of partcular concern for the asset allocaton problem due to the combnaton of contnuous and dscrete varables. In partcular, a lack of confguraton dversty was found to sgnfcantly ncrease the chances of convergence to a local mnmum. It was found that once the populaton was comprsed manly of a sngle confguraton, n most cases the search converged to the optmal set of ntercept and break-off tmes for that confguraton, rather than the globally optmal soluton. Once populaton confguraton dversty drops, t s unlkely to ncrease through recombnaton, mutaton, and eltst selecton. After eltst selecton, the remanng selecton methods were chosen to help mantan populaton dversty wth partcular consderaton gven to confguraton dversty. Of the μ solutons chosen by the selecton process, some are chosen usng eltsm whle the remander are chosen usng dversty quotas, populaton caps, and stochastc tournament. The porton of the new populaton chosen by each method s controlled by algorthm parameters. Each selecton method wll be dscussed n the followng sectons Eltsm Snce the selecton process uses (μ+λ)es, all selecton operatons draw solutons from the unon of the parent and chld populatons. The eltsm operaton selects the most ft solutons from the unon populaton to survve to the next generaton. The N EL

120 108 solutons wth the lowest ftness cost are added to the new populaton for the next generaton. N EL s a parameter of the algorthm whch specfes the number of solutons selected each generaton usng eltsm. Due to the fact that all recombnaton and mutaton operatons are performed at random, t s possble for a chld to be produced that s an exact copy of ts parent. When a copy s produced of the most ft soluton, the resultng offsprng s also the most ft soluton. Ths means both wll be selected under eltsm. Left unchecked ths wll produce a large number of duplcates of the solutons wth the best ftness. Allowng ths can quckly drop populaton dversty. In order to avod ths stuaton, all solutons selected usng eltsm must be unque wthn the new populaton. Ths allows only copy of each soluton to be selected each generaton usng eltsm Dversty Quotas The dversty quota selecton operaton s used to forcbly mantan confguraton dversty wthn the populaton. It does ths by forcng the selecton of solutons from multple confguratons. As prevously defned, the confguraton of a soluton s the set of all actve flags. Therefore, solutons wth the same confguraton represent solutons nvolvng the same combnaton of nodes. The dversty quota selecton operaton s controlled by two parameters, N Q and Q. The frst parameter, N Q, controls the number of quotas. The number of quotas s the mnmum number of unque confguratons whch must be present n the new populaton. The second parameter, Q, controls the sze of the

121 109 quotas. The quota sze s the mnmum number of solutons wth each confguraton whch must be present n the new populaton. The dversty quota process begns by determnng the N Q best confguratons n the current populaton. Ths s done by dentfyng the N Q solutons wth the lowest ftness costs, where each of these solutons has a dfferent confguraton. The confguratons of these solutons, c, are used as the quota confguratons. Once the confguratons are dentfed, Q solutons wth each confguraton are selected to survve to the next generaton. For all { 1 } LN Q, the Q most ft solutons wth confguraton c are selected to survve. If there are less than Q solutons n the current populaton wth confguraton c, prevously chosen solutons are selected agan, untl the quota s met. Overall, Q N Q solutons are selected usng dversty quotas. In addton to provdng a lmted measure of dversty, the dversty quota selecton operaton encourages the exploraton of promsng confguratons. Ths s due to the fact that quota confguratons are selected solely on the bass of ther most ft member. Ths selecton s ndependent of the number of solutons wth that confguraton or the ftness of those solutons. If a sngle soluton for a gven confguraton demonstrates a low ftness cost, that confguraton shows promse, even f t s underrepresented or poorly represented n the current populaton. Dversty quotas encourage exploraton of a promsng confguraton by selected a dsproportonately hgh number of ts members relatve to the number and overall qualty of those solutons.

122 Populaton Caps 110 Although not an actual selecton procedure, populaton caps are used by the selecton process to encourage dversty. Despte havng the same objectve as dversty quotas, populaton caps perform the opposte functon. Rather than enforcng a mnmum number of solutons per confguraton, as wth dversty quotas, populaton caps lmt the maxmum number of solutons per confguraton n a populaton. However, unlke dversty quotas, populaton caps are enforced over all confguratons, rather than a lmted number each generaton. The populaton caps encourage confguraton dversty by preventng a majorty of the solutons n a generaton from usng the same confguraton. The populaton caps are enforced durng stochastc tournament selecton. The populaton caps wll not allow any soluton to be selected whch ncreases the number of solutons n the new populaton usng a gven confguraton to greater than C pop, the maxmum number of solutons allowed for each confguraton Stochastc Tournament Once eltst selecton and dversty quota selecton are complete, the remanng solutons n the new populaton are selected usng stochastc tournament. For each soluton to be selected usng stochastc tournament, N entres solutons are chosen at random from the unon populaton. The most ft of these solutons s chosen wth probablty p tournament. In the event that the most ft soluton s not chosen, one s chosen at random from the set of N entres solutons. Stochastc tournament selecton s repeated untl the new

123 111 populaton s flled. Once the new populaton s formed, the next generaton begns and the above processes are repeated. Table 4-4 shows an example of the selecton process. The table on the left contans the unon of the parent and chld populatons of the current generaton, sorted by ncreasng ftness cost. The table on the rght shows the new populaton for the next generaton created by the selecton process. For each soluton, only ts ndex n the unon populaton and ts confguraton are shown. In ths table, an abbrevated decmal representaton s used for the confguraton. If the actve flags are treated as a bnary number, then the confguraton s expressed as the decmal equvalent of that number. For example, f a soluton contans 5 nodes and nodes 1, 3, and 4 are actve, ths s expressed as confguraton = Solutons selected usng eltst selecton are shown n red and marked wth a crcle. Despte ther low ftness costs, duplcate solutons such as are not selected usng eltsm. Solutons selected usng dversty quotas are shown n green and marked wth a damond. The remander of the solutons n the new populaton are selected usng stochastc tournament, shown n blue and marked wth a square. Solutons whch have prevously been selected usng eltsm or dversty quotas may be selected agan by stochastc tournament. Despte the relatvely large number of solutons n the unon populaton wth confguraton 13, only a lmted number are selected to survve due to the effect of populaton caps.

124 11 Table 4-4: Selecton process example Unon populaton generaton k (μ+λ) solutons New populaton generaton (k+1) μ solutons Rank Index Confg Selecton Selecton Index Confg N EL M M Q μ+λ M M Q μ+λ-8 11 Q N Q M M M M M Q μ+λ- 1 M M M μ+λ-6 8 p μ+λ p+1 μ+λ p+ μ+λ-7 μ+λ-9 18 μ - Q N Q - N EL p+3 μ+λ-6 8 M M p+4 μ+λ p+5 μ+λ p+6 μ+λ-3 13 p+7 μ+λ- 1 3 p+8 μ+λ-1 13 p+9 μ+λ 13

125 4.4.7 Search Scope 113 An mportant consderaton n locatng the globally mnmum soluton s that of search scope. The scope of the evolutonary search s used to descrbe the extent to whch the search explores the soluton space. A search whch s sad to be narrow n scope wll make small and nfrequent changes to the populaton of solutons, explorng wthn a relatvely small area of the soluton space. A search whch makes only small changes n solutons s lkely to fnd a nearby local mnmum. However, the same search s unlkely to determne whether or not ths local mnmum s the global mnmum. A search whch s sad to be broad n scope wll make large, frequent changes to the populaton, explorng a large area of the soluton space. By explorng more of the soluton space, the search s more lkely to locate the global optmum than a search of narrower scope. However, snce large changes n the populaton are made durng a broad search, t s much slower to reach a local mnmum than a narrow search. In the case of the asset allocaton problem, a broad search s necessary to dentfy the optmal confguraton as well as the optmal orderng of node responses wthn ths confguraton. Conversely, a narrow search s necessary to determne the optmal ntercept and break-off tmes. A search usng ether of these scopes alone was deemed to be nsuffcent. A broad search would take too many generatons to reach the mnmum, whle a narrow search would lkely settle at a suboptmal local mnmum. One possble soluton to ths problem was to choose a search scope whch balanced the requrements of a broad search and those of a narrow search. However, as ths partally satsfes both requrements, t also partally dssatsfes both requrements

126 114 and suffers the problems of both. Rather than attemptng to strke a balance, the approach presented here changes the scope as the search progresses. Thus, the search begns wth a broad scope, attemptng to locate the optmal confguraton. As the search progresses, the scope narrows n order to determne the optmal ntercept and break-off tmes for the canddate solutons Global Factor In an Evolutonary Strateges algorthm, the scope of the search s controlled by the parameters of ts operatons. Ths means that n order to alter the search scope, the parameters governng the recombnaton, mutaton, and selecton operatons must be modfed as the search progresses. Begnnng at some predetermned value, each parameter must change as the state of the search changes. It was decded that t would be too complex to create and calbrate ndependent mappngs relatng each parameter to the current search state. Dong so would also make t dffcult to later adjust the rate at whch the search scope changed. Instead, a sngle parameter s used to represent the current search scope. All other ES parameters are set through mappngs to ths sngle parameter. Ths parameter s referred to as the global factor. The global factor represents the current search scope on a scale of zero to one. At the begnnng the ES search, the global factor s set to one, representng the broadest scope that s to be used. As the search progresses, the global factor decreases, changng the search from a broad search to a narrow one. By the end of the search, the global factor s zero, representng the narrowest scope that s to be used n the search.

127 115 Each of the ES parameters s determned by the global factor through an affne mappng. Thus, the values whch a parameter may take are specfed by an ntal and fnal value for each parameter, P nt and P fnal. The value of parameter, P, s calculated based on these lmts and the global factor, gf, usng P nt fnal ( gf ) ( gf ) P + ( gf ) P = 1. (4.90) The lmts for each parameter are chosen such that a broad search takes place ntally and a narrow search takes place near the end of the algorthm. Choosng parameters for a broad search requres usng larger mutatons and favorng operatons whch result n greater exploraton, partcularly of new confguratons. These operatons nclude splt, combne, swtch, exchange, algn, and staton swap. Addtonally, the number of dversty quotas s ncreased, forcng exploraton of more confguratons. As the search moves toward a narrower scope, the probabltes of the above operatons decrease, as well as the number of quotas used. As the probabltes of these operatons decrease, those of the shft and slde operatons, whch perform local exploraton, ncrease. In addton, the standard devaton of ntercept and break-off tme mutatons decreases as the search focuses on the mmedate neghborhood of current solutons. As the number of quotas decreases, the sze of the remanng quotas ncreases to encourage further exploraton of the most promsng confguratons. Fnally, the probablty of selecton of the most ft soluton n stochastc tournaments s ncreased, whch decreases exploraton and speeds convergence. A summary of the drecton of change and lmts of each parameter, along wth a short descrpton, s presented n Table 4-5.

128 Table 4-5: ES parameter changes wth changng global factor 116 Parameter Drecton 5 Intal Value Fnal Value Descrpton Recombnaton parameters: p recom Probablty that a chld wll be formed from more than one parent Mutaton parameters: σ Standard devaton of mutatons appled to ntercept/break-off tmes, as a fracton of doman pmut actve Mutaton probablty for actve flags pmut tme Mutaton probablty for ntercept/break-off tmes pmut splt Probablty that splt operaton s appled pmut combne Probablty that combne operaton s appled pmut swtch Probablty that swtch operaton s appled pmut exchange Probablty that exchange operaton s appled pmut slde Probablty that slde operaton s appled pmut shft Probablty that shft operaton s appled pmut algn Probablty that algn operaton s appled pmut swap Probablty that swap operaton s appled ShftFactor Upper bound on shft dstance as a fracton of coverage nterval n the drecton of the shft SldeFactor Upper bound on slde dstance as a fracton of coverage nterval Selecton parameters: N EL Number of solutons selected usng eltsm N Q mn: N -1, 8 mn: N -1, Number of confguratons used n dversty quotas Q Number of solutons n each quota C pop + max: 50/ N, 1 60 Maxmum number of solutons n the populaton per confguraton p tournament Probablty of selectng the best soluton n each stochastc tournament 5 Drecton n whch each parameter changes as the global factor decreases,.e. as the search progresses.

129 Search Rate 117 The search rate of the evolutonary algorthm s the rate at whch the global factor changes from ts ntal value of one to ts fnal value of zero. Ths s equvalent to the rate at whch the search changes from a broad scope to a narrow scope. For a gven problem, ths search rate s constant. The rate s chosen such that the global factor reaches zero at a gven generaton, as shown n Fgure Global factor gen zero Generaton Fgure 4-9: Global factor search rate The generaton at whch the global factor wll reach zero, gen zero, s chosen based on the number of moble nodes n the network, N, accordng to gen ( N ) = G G. (4.91) zero fxed + node As the number of moble nodes ncreases, so does the length of the search. Ths decreases the search rate, keepng the global factor hgh for more generatons. Ths allows the search more tme to explore the ncreased number of confguratons resultng from the ncrease n the number of moble nodes.

130 4.4.8 Search Termnaton 118 The evolutonary search termnates when two crtera are satsfed. These crtera are that (1) the search s narrow n scope and () the search s no longer makng suffcent progress. The frst crteron s met when the global factor reaches zero. When ths occurs, the search s at ts narrowest possble scope. The second crteron s observed by montorng decreases n bestftness, the best ftness value from each generaton. When the best ftness value fals to decrease by Conv threshold over the Conv wndow most recent generatons, then the search s sad to have converged. For the current generaton, gen, ths condton s gven by ( gen Convwndow ) bestftness( gen) Convthreshold bestftnes s <. (4.9) Due to the use of eltst selecton, the functon bestftness s non-ncreasng. Therefore the dfference term n Equaton 4.9 s always greater than or equal to zero. When the global factor s zero and the search has converged, then the evolutonary search stops and the most ft soluton of the fnal generaton 6 s chosen as the result. 4.5 Smulaton Results Ths secton wll demonstrate the performance of the presented ES algorthm for asset allocaton problems through a smulated unmanned survellance network. Multple trals wll be used to evaluate the ftness and relablty of the generated solutons. The frst secton wll summarze the setup and executon of the tests. The next secton wll 6 Due to eltsm, ths s the most ft soluton of the entre search.

131 119 demonstrate the effects of the varous algorthmc mprovements covered n Secton 4.4. Then the ES-based algorthm wll be compared wth an assortment of determnstc allocaton methods. Fnally, the operaton of the algorthm wll be examned when gven a problem wth more advanced condtons, such as track changes and currents Smulaton Methodology A smulaton was developed to evaluate the performance of the ES algorthm as a centralzed controller n an unmanned survellance network. The smulaton models the moble nodes and the contact durng the allocaton phase of the network. Because the vehcles are not drectly controlled by the central controller, no vehcle dynamcs are modeled. Ths model s acceptable snce the control approach s desgned for networklevel, rather than vehcle-level, control. Addtonally, n the sparse network under consderaton, the vehcles wll spend lttle tme maneuverng relatve to the tme spent travelng n a straght lne. Each asset allocaton smulaton begns wth the smulated detecton of a contact and generaton of a track. Ths track contans the current poston, headng, and speed of the contact. The current state 7 of each moble node, track parameters, and boundary nformaton are then fed to the ES algorthm, whch generates a soluton to the asset allocaton problem. Ths soluton s then converted to a seres of wayponts whch are used to command the moble nodes. As the smulaton contnues, asset allocaton problem 7 Includes current poston, energy levels, maxmum possble speed, staton assgnment, and local current nformaton.

132 10 parameters are updated and resubmtted to the ES algorthm as necessary. If the track has changed by more than a small amount, then a new soluton s generated and waypont recommendatons are updated. To detect a track change, the current track s compared to the track used to generate the prevous soluton. A track s consdered to have changed f the headng changes by more than one degree, the speed changes by more than 0.1 km/hr, or the contact moves more than 100m from ts projected poston at the current tme 8. The ES algorthm uses the parameter values gven n Secton to control recombnaton, mutaton, and selecton operatons. Parameters not controlled by the global factor are set accordng to Table The poston the contact was projected to be at accordng to the prevous track.

133 11 Table 4-6: ES parameter values Parameter Value Descrpton Secton Objectve functon parameters MP.0 Maxmum low energy penalty LP 0. Low energy penalty threshold c LP 0.01 Exponental penalty value at LP w f.0 Free tme weght w e 0. Energy cost weght w b 0.9 Energy balance weght w p 1.0 Max speed penalty weght pen fxed 4.0 Max speed penalty fxed cost pen varable 4.0 Max speed penalty varable cost T horzon 5.0 hr Fxed tme horzon for runtme calculaton Search termnaton parameters G fxed 100 Mnmum number of generatons G node 85 Number of addtonal generatons per node Conv wndow 0 Number of generatons used to evaluate convergence Conv threshold 1e-4 Maxmum mprovement for convergence Selecton and populaton parameters μ 100 Parent populaton sze 4.3, λ 100 Chld populaton sze N entres 3 Number of entres per stochastc tournament An example of an asset allocaton problem s shown n Fgure The contact s shown at the moment of track formaton, wth all nodes holdng poston at ther ntal statons. The track s projected usng the constant velocty assumpton. The parameters for ths problem are lsted n Table 4-7.

134 Moble node Node ndex Staton Contact Track y [km] x [km] Fgure 4-30: Example asset allocaton problem Table 4-7: Example asset allocaton problem parameters Parameter Descrpton Value v n Maxmum speed of node [m/s].8 v o Local current speed for node [m/s] 0 v o Local current headng for node [deg] 0 (clockwse from North) hrsreman Remanng runtme for node [hr] 100 v c Contact speed [m/s]. ψ Contact headng [deg] (clockwse from North) 0 When ths example problem s gven to the ES algorthm, a soluton s generated. The moble node response specfed by ths soluton s shown n Fgure 4-31 after the contact has left the area of nterest. The thck colored lnes ndcate the paths taken by the

135 13 moble nodes as the contact passed through the area of nterest. In ths soluton, the coverage task was dvded between four nodes n order to balance energy consumpton. Nodes 3 and 4 were not ncluded n the response due to the large dstance they would be requred to travel and the hgh energy cost assocated wth ths acton. In ths soluton, the closest node ntercepts the contact as soon as possble and no coverage gaps are left, resultng n the lowest possble amount of free tme for ths allocaton problem Moble node Moble node path Contact Track 1 y [km] x [km] Fgure 4-31: Example asset allocaton problem response

136 4.5. Performance Effects of ES Operators 14 In order to examne the effects of the mprovements made to the ES allocaton algorthm, the algorthm was run multple tmes gven the same nputs. These tests were repeated for dfferent test cases, each usng dfferent settngs of algorthm parameters. All tests n ths secton used the example problem presented n Fgure 4-30 and Table 4-7. Three test cases are explored. The frst test case s referred to as normal settngs. In ths setup, all operatons are used and algorthm parameters are set as descrbed n prevous sectons. In the lmted operatons test case, all non-standard mutaton and selecton operatons are dsabled. Ths means the swtch, splt, combne, algn, exchange, shft, and slde operatons are dsabled. In addton, selecton dversty quotas and populaton caps are not used. Ths means the evolutonary search must rely on smple mutaton, recombnaton, eltst selecton, and stochastc tournament. The fnal test case s fxed global factor. In ths test case, the global factor s fxed at 0.5 for the course of the search. Ths means the algorthm parameters do not change durng the search, keepng the scope fxed between a broad and narrow search. Fgure 4-3 shows the convergence rate of a sngle run of the ES algorthm for each of the three test cases. As can be seen n the fgure, the algorthm converges to the lowest ftness cost when usng normal settngs. When lmted operatons are used, the search converges to a much hgher ftness value. Wthout the ad of the addtonal mutaton and selecton operatons, the search quckly settles on a suboptmal confguraton of actve nodes, whch results n a hgher ftness value. When usng a fxed global factor, the search ntally converges more quckly. However, near the end of the

137 15 search the ftness contnues to mprove when usng normal settngs whle t settles at a hgher value when usng a fxed global factor. Overall, the normal settngs result n the best soluton. 0.8 Normal settngs Lmted operatons Fxed global factor Ftness Generaton Fgure 4-3: ES convergence usng dfferent parameter settngs To compare the average performance of the ES algorthm wth and wthout mprovements, tests were conducted usng the three test cases dscussed above. For each test case, 300 solutons were generated for the test problem. Fgure 4-33 shows the mean convergence rate for each of the test cases. As wth the sngle run results, the addton of the global factor s shown to slow convergence but mprove the fnal ftness value. And agan, the use of addtonal mutaton and selecton operators s shown to dramatcally mprove the fnal ftness value.

138 Normal settngs Lmted operatons Fxed global factor 0.76 Ftness Generaton Fgure 4-33: Mean ES convergence usng dfferent parameter settngs A scatter plot of fnal ftness value versus runtme for the 900 trals s shown n Fgure 4-34, wth a close-up of the majorty of the trals shown n Fgure The plot shows that whle the search can arrve at a near-optmal soluton usng a fxed global factor (green trangles), t does so more relably usng normal settngs wth a varable global factor (blue dots). Ths can be more easly seen n Fgure 4-36, whch shows a hstogram of fnal ftness values for solutons generated usng normal and fxed global factor settngs. The solutons marked wth red squares n Fgure 4-34 represent those generated wth lmted operatons. These solutons can easly be seen to be greatly sub-optmal and very unrelable. Each successve gatherng of ponts about a front along the ftness axs represents a dfferent sub-optmal confguraton. Agan, a confguraton s the set of all actve flags n a soluton, representng whch nodes wll be nvolved n the response.

139 17 Wthout the addtonal mutaton and selecton operators, the search quckly settles on a sub-optmal confguraton, whch t never leaves. A hstogram showng the dstrbuton of fnal confguratons n the results of the trals for each of the test cases s shown n Fgure The confguratons are represented by decmal equvalents of the actve flags when treated as a bnary number. For example, n the soluton shown n Fgure 4-31, nodes 1,, 5, and 6 are actve. Ths confguraton s represented by [ ] = The hstogram shows that for normal settngs and fxed global factor, the majorty of solutons belong to the optmal confguraton, wth the normal settngs performng slghtly better n ths regard. Wthout the addtonal mutaton and selecton operatons, none of the resultng solutons belong to ths confguraton, nstead settlng at varous sub-optmal confguratons Normal settngs Lmted operatons Fxed global factor Runtme [sec] Inset Ftness value Fgure 4-34: Ftness vs. runtme scatter plot usng dfferent parameter settngs for the example problem

140 Normal settngs Lmted operatons Fxed global factor 95 Runtme [sec] Ftness value Fgure 4-35: Ftness vs. runtme scatter plot for dfferent parameter settngs (nset) 100 Normal settngs Fxed global factor 80 Frequency Ftness Fgure 4-36: Ftness hstogram for normal and fxed global factor settngs

141 Normal settngs Lmted operatons Fxed global factor 00 Frequency Confguraton Fgure 4-37: Confguraton hstogram for dfferent parameter settngs The effect of the addtonal mutaton and selecton operatons can also be seen through the use of dversty maps. Dversty maps show the nternal state of the evolutonary search. Each row of the map represents the number of solutons n the populaton of a sngle generaton usng each of the possble confguratons. Plotted over the course of the entre search, a dversty map shows how the dstrbuton of confguratons wthn the soluton populaton changes as the search progresses. The dversty map for a sngle run of the ES algorthm wth lmted operatons, for the example problem, s shown n Fgure 4-38(a) and wth normal settngs n Fgure 4-38(b). These maps show that more exploraton of confguratons occurs wth normal settngs than wth the addtonal mutaton and selecton operatons removed. Usng lmted operatons, the search quckly settles on sub-optmal confguraton 33, wth most of the

142 130 populaton usng ths confguraton. Wth normal settngs, many confguratons are explored, eventually settlng on confguraton 51. Even though the full potental of ths confguraton s not dentfed untl later n the search, the algorthm s forced to explore t early on through dversty quotas. Ths allows t to eventually become the domnant confguraton n the populaton and the confguraton of the fnal soluton Generaton Generaton Number of solutons Confguraton (a) Confguraton Fgure 4-38: Dversty maps for lmted operatons (a) and normal settngs (b) (b)

143 4.5.3 Algorthm Comparson 131 In ths secton, the performance of the ES-based allocaton algorthm presented n ths chapter s compared wth that of two determnstc allocaton methods. The solutons produced by the competng allocaton methods are evaluated usng the same ftness functon as the ES algorthm. The two determnstc allocaton methods used are Vorono allocaton and k-best allocaton. The Vorono allocaton approach uses Vorono dagrams to choose whch nodes to actvate and to assgn coverage ntervals to those nodes. A smlar approach was used n [60] to perform fxed sensor actvaton n a densely populated network. In the Vorono approach, each node assumes survellance responsblty for the target as t passes through that node s Vorono regon. The Vorono regon for each node s defned based on the postons of all nodes accordng to j ( ) = { x : x x x x, j } V x, (4.93) n n where x n s the poston of the th node. If the contact track passes through the Vorono regon of node, node s set to actve status and assgned a coverage nterval correspondng to the porton of the track passng through the Vorono regon of node. Ths process s best llustrated by an example allocaton usng the Vorono method, as shown n Fgure The track and moble node paths are represented as n the prevous example, and the boundares of the Vorono regons are marked wth dashed lnes. The Vorono allocaton method has the advantage that all portons of the track are covered by the closest possble moble node. The Vorono approach gves no consderaton to any factors other than contact track and moble node poston. n

144 y [km] 0-1 Fgure 4-39: Vorono allocaton example x [km] The k-best allocaton approach dvdes the survellance task between the k nodes closest to the track. The nodes closest to the track are defned as those whch have the shortest closest pont-of-approach (CPA) dstance, d CPA. An example problem showng node postons, contact track, and CPA dstances s shown n Fgure The CPA dstance for node s gven by dcpa = [ x ] ( ψ + π ) ( ) ψ + π cos x c n, (4.94) sn where ψ s the contact headng. The tme at whch the contact wll reach the closest pontof-approach to node s gven by tcpa = 1 v c [ x x ] n c cos sn ( ψ ) ( ) ψ. (4.95)

145 x n 6 x n 9 9 d CPA x c 3 x n 6 d CPA x n 8 8 d CPA y [km] x n 1 d CPA 1 d CPA x n 4 4 d CPA 3 d CPA 7 d CPA 5 d CPA x n 3 x n 7 x n Fgure 4-40: k-best allocaton example x [km] The nodes are allocated by frst choosng the k nodes wth the smallest CPA dstance (d CPA ) and then orderng them accordng to the tmes the contact wll reach them (t CPA ). The ndces p specfy the k nodes selected n ths manner. The nodes wth ndces p are specfed as actve and ther coverage ntervals are set accordng to t p1 nt t pn brk t pk brk = mn p nt p1 ( BT ) 1 pn pn+ 1 = ( tcpa + tcpa ) pk ( B ) n+ 1 = t 1 n < k. (4.96) = max T An example soluton usng k-best allocaton s shown n Fgure In ths problem, nodes 1, 4, and 8 are closest to the track. These nodes are set to actve status and assgned handoff locatons mdway between each CPA. No consderaton s gven to any

146 134 factors other than node poston and contact track. Despte ths, k-best allocaton has the advantage that the closest nodes to the track always respond, lmtng energy consumpton. In order to compare the performance of the ES-based approach wth that of the determnstc methods, each method was appled to dentcal problems and the ftness of the resultng solutons was compared. Ths process was repeated over 400 trals, each tme usng a new random problem. The parameters n Table 4-7 were used wth the excepton of target headng. Target headng was calculated by choosng a unformly dstrbuted random target entrance locaton along the northern (upper) border and a random target ext locaton along the southern (lower) border. The number of moble nodes was unformly randomly selected from between 4 and 9 for each tral. The postons of these nodes were unformly randomly chosen wthn the area of nterest for each tral. Examples of problems contanng random node postons and tracks are shown n Fgures 4-39 and For each tral the ES-based method, the Vorono method, and the 1-Best, -Best, and 3-Best methods were appled. In each tral, the soluton generated by each method was evaluated for ftness. In order to compare the results, the percentage dfference between each method s ftness and that of the ES method was computed n each tral. If ths dfference s greater than zero, then the compettor produced a soluton wth a hgher ftness value that s less ft than the ES soluton. If ths dfference s less than zero, then the compettor produced a soluton wth a lower ftness value that s more ft than the ES soluton.

147 135 The performance comparson results of 400 trals are summarzed n Tables 4-8 and 4-9. Table 4-8 shows the number of trals n whch each determnstc approach generated a better soluton than the ES-based approach. The frst column shows the amount by whch the solutons on each row were better than the ES soluton. The next four columns show the results for each of the determnstc allocaton methods. The last column shows the results when usng the best determnstc approach. Ths approach takes the results of all four determnstc methods and selects the best result n each tral. Table 4-9 shows the number of trals n whch each determnstc approach generated a worse soluton than the ES-based approach. These tables show that whle the determnstc methods performed better than the ES approach n a small number of cases, most of these cases showed less than 0.01% mprovement. An mprovement of 0.01% s, for all practcal purposes, the same soluton. On the other hand, n the majorty of cases the determnstc approaches performed worse, often by a sgnfcant margn. In many cases, ftness values were 5-10% hgher, whch s a sgnfcant dfference n terms of physcal soluton nterpretatons. In some cases, ftness values were as much 100% hgher than those of the ES solutons. Table 4-8: Determnstc allocaton method comparson results superor solutons Number (percentage) of trals wth better solutons than ES Dfference from ES ftness less than Vorono allocaton 1-Best allocaton -Best allocaton 3-Best allocaton Best determnstc -0.01% 0 (0.00%) 37 (9.5%) 0 (5.00%) 13 (3.5%) 70 (17.50%) -0.1% 0 (0.00%) (0.50%) 7 (1.75%) 8 (.00%) 17 (4.5%) -0.5% 0 (0.00%) 0 (0.00%) (0.50%) 5 (1.5%) 7 (1.75%) -1.0% 0 (0.00%) 0 (0.00%) 0 (0.00%) 3 (0.75%) 3 (0.75%)

148 Table 4-9: Determnstc allocaton method comparson results nferor solutons Dfference from ES ftness greater than Number (percentage) of trals wth worse solutons than ES Vorono 1-Best -Best 3-Best allocaton allocaton allocaton allocaton 136 Best determnstc 0.01% 400 (100.0%) 358 (89.50%) 379 (94.75%) 386 (96.50%) 33 (80.75%) 0.1% 399 (99.75%) 358 (89.50%) 368 (9.00%) 381 (95.5%) 307 (76.75%) 1% 399 (99.75%) 356 (89.00%) 318 (79.50%) 338 (84.50%) 16 (54.00%) 5% 61 (65.5%) 334 (83.50%) 17 (54.5%) 48 (6.00%) 56 (14.00%) 10% 165 (41.5%) 95 (73.75%) 148 (37.00%) 171 (4.75%) 14 (3.50%) 0% 66 (16.50%) 196 (49.00%) 94 (3.50%) 11 (8.00%) 3 (0.75%) 50% 7 (1.75%) 90 (.50%) 35 (8.75%) 55 (13.75%) 0 (0.00%) 100% 1 (0.5%) 3 (0.75%) 5 (1.5%) 19 (4.75%) 0 (0.00%) 00% 0 (0.00%) 0 (0.00%) 1 (0.5%) 5 (1.5%) 0 (0.00%) Another means of examnng the algorthm performance comparson results s the box plot shown n Fgure 4-41(a). The box plot agan shows the percent ncrease n ftness cost over ES for the dfferent determnstc allocaton methods. An ncreased ftness cost represents a less ft (nferor) soluton. The boxed regon covers the nd and 3 rd quartles, whle the whskers extend from the box to samples up to the length of the nterquartle regon away. Outlers are marked wth pluses. A close-up of the nterquartle regons s shown n Fgure 4-41(b). The box plot shows that n the vast majorty of cases, the ES-based approach performs better than any one determnstc method. In addton, t also shows that n most cases, the ES-based approach performs better than all of the determnstc methods combned.

149 Percent ncrease over ES ftness cost Percent ncrease over ES ftness cost Vorono 1-Best -Best 3-Best Best det. Vorono 1-Best -Best 3-Best Best det. (a) (b) Fgure 4-41: Determnstc allocaton method ftness comparson results all trals (a) and close up of nterquartle regons (b)

150 4.5.4 Advanced Condtons 138 In ths secton, the response of the ES-based allocaton algorthm to a more complex problem s demonstrated. Ths ncludes changes n target headng, ocean currents, and node staton swappng. Ths example problem conssts of 7 nodes wth node postons and ntal track as shown n Fgure 4-4. The boldface numbers ndcate the ndex of each moble node, whle the decmal numbers show the hours of remanng runtme for each node. The arrows show the headng and magntude of the local current for each node. The non-poston parameters of the problem are lsted n Table Parameter descrptons can be found n Table Moble node Moble node path Contact Track Local current Node ndex Remanng runtme y [km] x [km] Fgure 4-4: Advanced asset allocaton example

151 139 Table 4-10: Advanced asset allocaton example parameters Node ndex v n [m/s] v o [m/s] v o [deg] v c [m/s]. The resultng network response usng the ES allocaton algorthm s shown n Fgure The dashed lnes show the allocated survellance ntervals as defned by the ES soluton. Due to the fact that node 4 ntally has more energy remanng than the other nodes, t receves a larger porton of the survellance burden. Ths helps balance energy levels across the network wthout consumng too much energy overall. The sold, colored lnes show the paths of the moble nodes up to the current tme. These paths show how node 6 uses the current to assst ts return trp to ts assgned staton. After completng ts survellance leg, node 6 moves approxmately north under ts own power. When t reaches a specfed pont, t then allows tself to drft freely. The current then carres the node the remander of the way to ts staton. Nodes whch are not used for survellance reman at ther statons. These nodes ntermttently drft then reposton themselves as necessary to keep staton.

152 Moble node Moble node path Contact Track Node ndex Coverage assgnment Remanng runtme y [km] x [km] Fgure 4-43: Asset allocaton example - ntal moble node response After the nstant shown n Fgure 4-43, the target changes headng. The ES algorthm s then used to generate a soluton for the new problem parameters. The resultng moble node response s shown n Fgure After the headng change, the target s movng towards node, whch s lower on energy than node 4. In order to conserve the energy of node, node 4 contnues to mantan survellance of the target. To prevent the entre energy burden from beng placed on node 4, node swaps statons wth node 4. Ths requres node to travel a short dstance to ts new staton (whle utlzng ocean currents), and allows node 4 to return to a staton whch s much closer to ts breakoff pont.

153 Moble node Moble node path Contact Track Node ndex Remanng runtme y [km] x [km] Fgure 4-44: Asset allocaton example moble node response after target headng change

154 14 Chapter 5 Drected Search As presented n Secton 3.4.1, the drected search problem s that of choosng search paths for one or more moble nodes n order to locate a target travelng along an uncertan track. These search paths should allow for a hgh probablty of target detecton whle requrng the expendture of as lttle energy as possble. Ths chapter wll present a detaled formulaton for ths problem, as well as an approach for generatng solutons. The chapter begns wth the formulaton of the drected search problem, ncludng the representaton of search paths and the objectves aganst whch potental paths are evaluated. The next secton presents a method of expressng track uncertanty and detecton probablty usng Gaussan dstrbutons. Ths secton also descrbes a method for estmatng total detecton probablty for multple paths. The followng secton gves an overvew of prevous approaches for search path generaton. Next, the two-tered soluton approach for drected search problems s presented. Lastly, performance results are gven, demonstratng the effectveness of the approach. 5.1 Problem Formulaton The defnton of the drected search problem gven thus far allows for a wde range of possble nterpretatons. Many possble defntons of an optmal search path could be gven. In ths secton, the formulaton of the problem s defned, ncludng the

155 defnton of search paths and the objectve functon used to evaluate multple search paths Representaton of Search Paths A soluton to the drected search problem conssts of one feature, a search path for each node nvolved n the search. These search paths specfy the path each node should follow as t searches for the target. Due to the autonomous nature of the moble nodes, approxmate search paths, rather than exact paths, are gven. These approxmate paths take the form of a seres of wayponts for each node. Each node s responsble for autonomously choosng ts own path between wayponts. Each waypont conssts of a poston, x n j, and a tme, t n j, by whch the node should reach ths poston. The ndex n ndcates to whch node the waypont belongs and the ndex j ndcates the sequence of the wayponts n the path. Each path conssts of a seres of such wayponts. Therefore, a drected search soluton s defned by n n n {, t } n { 1... N }, j { 1 N } S = x..., (5.1) j j where N DS s the number of nodes nvolved n the drected search and N n W s the number of wayponts n the path for node n. An example soluton of ths form s shown n Fgure 5-1. Ths example shows search paths for two nodes, along wth correspondng tmestamps (n hours) for each waypont. At the end of each search path, the node wll return to ts staton f t has not detected the target. DS W

156 144 y [km] Search path leg Waypont/tmestamp Startng Pont/start tme x [km] Fgure 5-1: Example drected search paths Optmzaton Objectves As prevously descrbed, the drected search paths must satsfy two objectves. Ths secton wll descrbe how these objectves are quantfed and combned nto a sngle ftness value. Ths ftness value s then used to represent the overall qualty of a set of search paths for a gven problem. The frst objectve s the maxmzaton of target detecton probablty. Ths objectve s represented by a sngle parameter, the total target detecton probablty, P total. Total target detecton probablty s the probablty that at least one node wll detect the target n the course of ts search. The defnton of a detecton, the representaton of track uncertanty, and the method n whch P total s evaluated are the subjects of Secton 5..

157 145 The second objectve s the mnmzaton of total expended energy. Lke the objectve functon for asset allocaton, energy consumpton s measured by node runtme. The more runtme a path requres, the more energy a node must consume to follow t. Usng ths defnton, the total energy consumpton cost s gven by the sum of the runtmes for every leg of every node s search path, accordng to E total = N DS W n= 1 n N j= 1 1 N DS n n n n ( t j+ t j ) = ( t n t1 ) 1. N (5.) W n= 1 Once both objectves are evaluated, the overall ftness of a soluton s calculated usng a weghted dfference, gven by F DS ( S ) wd Ptotal weetotal = (5.3) 5. Detecton Probablty Evaluaton In order to evaluate the total target detecton probablty, a number of factors must be consdered. Ths secton dscusses these factors, endng wth the calculaton of total target detecton probablty for a gven set of search paths. The secton begns wth a dscusson of Gaussan dstrbutons whch are used extensvely n the evaluaton process. The next secton presents the method used to model uncertan target tracks. Then, a sensor detecton model s presented along wth the means by whch the movement of the nodes s consdered. Fnally, the results of the prevous sectons are combned to form the total target detecton probablty.

158 5..1 Gaussan Dstrbutons 146 Gaussan dstrbutons are used extensvely n ths chapter to represent uncertan parameters. Ths s prmarly due to the fact that the lnear transformaton of a set of n jontly Gaussan random varables results n n jontly Gaussan random varables [61]. Ths property allows for a great deal of computatonal speedup, as dscussed n the followng sectons. In ths chapter, all random varables are (x,y) coordnate pars. Therefore, bvarate normal dstrbutons are used, defned by N 1 T 1 ( x, μ, Σ) = exp ( x μ) Σ ( x μ) π Σ 1 The vector x = [x y] T s the varable representng poston, μ s the mean value of x, and Σ s the covarance matrx of the dstrbuton. As proven n [61], Gaussan dstrbutons have the property that a lnear transformaton of n jontly Gaussan random varables results n n jontly Gaussan random varables. When a lnear transformaton, A, s appled to the jontly Gaussan random varables defned by (μ, Σ), the resultng jontly Gaussan random varables are defned by the parameters μ Σ A A 1. (5.4) = Aμ. T (5.5) = AΣA 5.. Uncertan Track Representaton The drected search problem s motvated by the antcpated uncertanty n the ntal target track. The form of ths uncertanty could vary, dependng on the ndvdual

159 147 fxed nodes and the target tracker used to fuse the sensor readngs from multple nodes. Dfferent systems wll have dfferent ways of reportng uncertanty. In order to create a unversal method of detecton probablty evaluaton, a sngle type of dstrbuton s assumed for track uncertanty n the drected search problem. The Gaussan dstrbuton was chosen for ths purpose, due to ts computatonal benefts. Uncertanty estmates usng non-gaussan dstrbutons must be approxmated usng a Gaussan dstrbuton. An uncertan target track conssts of two random varables, the ntal poston and velocty of the target. Each of these varables s represented by ts own Gaussan random varable. The ntal poston, X 0 = [X 0 Y 0 ] T, s the estmated poston of the target at tme t 0, the tme when the drected search wll begn. The probablty densty functon (PDF) for a specfc ntal target poston, x 0, s gven by P ntpos ( ) = N( x, Σ ) x 0 0, μntpos ntpos. (5.6) The random varable V = [V X V Y ] T represents the estmated velocty of the target, whle the varable v represents a specfc velocty. The PDF for the target velocty s gven by P vel ( ) = N( v, μ, Σ ) v. (5.7) vel vel These random varables allow the PDF for the target poston, X pos = [X pos Y pos ] T, at some future tme, t > t 0, to be calculated. Because both ntal poston and velocty are defned usng Gaussan dstrbutons, ths can be accomplshed wth lttle computatonal effort. In the case of determnstc ntal poston and velocty, the target poston at a future tme s gven by ( t) = x + v( t ) x. (5.8) 0 t 0

160 Smlarly, the future target poston can be calculated for Gaussan dstrbutons usng a lnear transform, A. Random varables for the target poston, X pos (t), and ts dsplacement, D 0, from ts ntal poston at tme t 0 are gven by the transform A, X Usng ths transformaton and applyng Equaton 5.5 gves the dstrbuton parameters for target poston and dsplacement at tme t, pos D ( t) X0 + V( t t0 ) = ( ) V t t0 I ( t t0 ) I X0 X0 = = 0 ( ) I V A t t V 0 0. (5.9) 148 μ Σ A A μ = A μ Σ = A 0 ntpos vel ntpos. (5.10) 0 T A Σvel Extractng the parameters for future target poston gves pos ( t) = μ ntpos + ( t t ) μ 0 vel () t = ΣntPos + ( t t ) Σvel μ pos. (5.11) Σ These smple transformatons allow the poston PDF for the target to be expanded to any future tme wth lttle computatonal overhead. The future target poston PDF, P pos ( ) (, t) = N x, μ ( t), Σ ( t) gves the probablty that the target wll be located at poston x at tme t. pos 0 x, (5.1) pos

161 5..3 Path Leg Detecton Probablty 149 Ths secton covers two aspects of the detecton probablty evaluaton process, the defnton of a detecton and accountng for the moton of the nodes. In order to determne the probablty of target detecton, the meanng of a detecton must be defned. In the drected search problem, a target s consdered detected f ts presence s captured by a sngle sensor readng of a sngle node. Whle a sngle detecton s not lkely to be suffcent to generate a track, the goal of the drected search problem s only to maxmze the probablty of an ntal detecton by a moble node. After ths ntal detecton, t s the responsblty of the moble node to autonomously nvestgate the target and form a more accurate track. In order to determne the lkelhood that a detecton wll occur at any gven moment, the sensor footprnt of the moble nodes must be defned. The sensor footprnt, P footprnt (d), defnes the probablty that a target located at a relatve dsplacement, d, from a moble node wll be detected by that node s sensors. Gven the target poston PDF at tme t, the sensor footprnt, and the node locaton x s, the nstantaneous detecton probablty for a sngle node at tme t can be found by ntegratng, P detect ( x t) = P ( x, t) P ( x x )dxdy s,. (5.13) pos footprnt The smplest sensor footprnt s one whch s determnstc, fxed-range, and omndrectonal. Usng ths model, a target s detected f t passes wthn a fxed dstance, r d, of the moble node. Such a determnstc footprnt s defned by s P footprnt ( d) 1 = 0 d d rd. (5.14) > r d

162 150 Whle ths s conceptually the smplest samplng footprnt, usng ths footprnt makes evaluatng nstantaneous detecton probablty dffcult. In order to calculate detecton probablty usng the determnstc footprnt, the ntegral r ( ) d π r cosθ Pdetect s t Ppos s t d dr r x, = x +, θ 0 (5.15) 0 snθ must be evaluated. Evaluaton of Equaton 5.15 requres ntegraton of a bvarate normal dstrbuton, whch has no exact closed form soluton. Although many numercal methods and other approxmatons exst [6, 63], all make requrements on the shape of the ntegraton area or do not have the runtme propertes of a closed-form soluton. Snce evaluatng P detect (x s,t) s part of the ftness functon for drected search solutons, runtme s an mportant consderaton. Even a relatvely fast numercal approxmaton technque would cause large ncreases n runtme over multple ftness evaluatons. Fortunately, the propertes of Gaussan random varables can agan be used to reduce the number of necessary computatons. If the moble node samplng footprnt s modeled usng a Gaussan dstrbuton, the nstantaneous detecton probablty can be easly calculated. Such a samplng footprnt s defned by P footprnt ( ) = N( d,0, Σ ) d β, (5.16) footprnt footprnt where the shape of the footprnt s characterzed by ts covarance matrx, Σ footprnt. Usng ths samplng footprnt, targets closer to the node are more lkely to be detected than those far away, whch s not an unrealstc assumpton. The scalng factor s set such that a node has a 100% chance of detectng the target f t drectly on top of t, footprnt Σ footprnt 1 β = π. (5.17)

163 151 Usng the lnear transformaton property of Gaussans dscussed above, the nstantaneous detecton probablty can be calculated by frst defnng a new random varable. The Gaussan random varable X detect (t) represents the poston of a moble node whch detects the target at tme t. In other words, a node located at X detect (t) at tme t wll detect the target. The varable X pos (t) represents the poston of the target at tme t, and the varable D footprnt represents the poston of the detected target relatve to the moble node. Usng these varables, the poston of the target when detected at tme t s gven by X pos (t) = X detect (t) + D footprnt. A lnear transformaton, B, can then be defned to calculate X detect (t), X D detect footprnt () t X ( t) = I = 0 D footprnt I X I D Usng Equaton 5.5, the parameters of the transformed varables can be calculated by Extractng the parameters of X detect (t) results n μ Σ B B μ Σ pos D μ pos = B μ Σ pos = B 0 detect detect ( t) footprnt footprnt () t X () t footprnt = B D. (5.18) The nstantaneous target detecton probablty for a node located at poston x s at tme t can now be relatvely easly calculated by evaluatng the PDF of X detect (t) at a sngle pont (the poston of the node, x s ) and multplyng by the samplng footprnt scalng factor, pos () t 0 T Σ footprnt B ( t) = μ pos ( t) () t = Σ pos () t + Σ footprnt pos footprnt. (5.19). (5.0)

164 P detect (, t) = N( x, μ ( t), Σ ( t) ) 15 x β. (5.1) s footprnt s detect detect Calculatng nstantaneous detecton probablty usng Equatons 5.0 and 5.1 gves the detecton probablty for a sngle node at a sngle poston and tme. To calculate total detecton probablty, ths value must be calculated at multple postons and tmes along the path of the node. Ths evaluaton must take place at a fnte number of ponts, and should use as few as possble n order to reduce evaluaton tme. In order to account for each leg n the node s search path, the nstantaneous detecton probablty s evaluated once for each leg. In order to account for the moton of the node durng the leg, the samplng footprnt s extended to cover the entre leg wth the node locaton set n the center of the leg. Although ths s only an approxmaton of the node s sensor coverage as t travels, for the purposes of creatng approxmate search paths, t s deemed to be suffcent. An example of a samplng footprnt extended over an entre leg s shown n Fgure 5-. The samplng footprnt s generated for each leg, j, and defned by ts covarance matrx, Σ n footprnt(j). The calculaton of ths covarance matrx s dscussed n Appendx A.1. The nstantaneous detecton probablty for each leg s then evaluated n n n usng ths footprnt, centered at the samplng locaton, x = ( x + x )/ n n n at the samplng tme, ( )/ t s( j) t j + t j+ 1 =. s ( j) j j+ 1, and evaluated

165 y [km] n n {x,tj+1 j+1 } n n {x,ts(j) s(j) } P footprnt (d) - samplng footprnt 55 {x j n,tj n } x [km] Fgure 5-: Search path leg samplng footprnt Total Detecton Probablty Wth the equatons developed n the prevous secton, the detecton probablty of a sngle path leg can be calculated. However, ths alone cannot be used to determne total detecton probablty because each path leg detecton probablty s calculated ndependently of all others. Allowng ths would result n all nodes followng the peak value of P pos (x,t) rather than searchng other possble areas. Instead, path hstory must be consdered so that a condtonal probablty s obtaned. Ths condtonal probablty ncludes the effects of the prevous measurements (legs).

166 154 Path hstory s consdered through the use of exclusons, areas whch have recently been searched and should be excluded n the near future. Excluson parameters are defned by generatng the samplng locaton, tme, and footprnt for all path legs of all nodes and set accordng to t k exc k exc k exc μ Σ β k exc = t n s( j) k ( texc ) k ( t ) exc = x = Σ = π Σ n s( j) n footprnt( j) 1 k k exc( texc ) n n { 1... N }, j { 1... N } The exclusons are sorted by samplng tme, such that k = 1 has the earlest samplng tme. The excluson functon s defned accordng to P k exc Ths functon defnes the probablty that a target located at poston x at tme t would have already been detected durng leg k. However, the parameters of Equaton 5. only defne the exclusons at ther samplng tmes. As tme passes, the excluded regon moves. For example, assume the track s determnstc wth velocty v and no detecton occurs at poston x s at tme t s. Then at tme t, the target wll not be found at poston x s +v(t-t s ). Snce the changes n the excluson are due to the movement of the target, the excluson expands accordng to the velocty PDF of the target. Usng the same reasonng as Equaton 5.11, the expanded excluson parameters are gven by μ Σ DS ( ) k k k (, t) = N x, μ ( t), Σ ( t) k exc k exc exc exc exc W. (5.) x β. (5.3) k k k ( t) = μexc( texc ) + ( t texc ) μvel k k k () t = Σexc ( texc ) + ( t texc ) Σvel. (5.4)

167 155 An example of a path leg excluson expandng wth tme s shown n Fgure 5-3 for one leg of one example path from Fgure 5-1. As tme passes, the excluson changes shape and expands based on the shape and sze of the target velocty PDF. As the excluson expands, ts peak magntude decreases. Ths represents the ncreasng uncertanty n the nformaton ganed from past samples as tme passes, due to the uncertanty n target velocty. After suffcent tme has passed, past samples gve vrtually no nformaton about future target postons. y [km] Search path leg Waypont Startng pont k t = t exc k t = t exc +1 k t = t exc (x,t) - Probablty of past detecton 75 k t = t exc P exc k x [km] Fgure 5-3: Excluson expanson over tme 0 Usng exclusons, condtonal path leg detecton probablty can be evaluated usng path hstory. The detecton probablty for each leg s multpled by the value of the excluson functon for all past legs evaluated at the current samplng locaton and tme.

168 Ths takes nto account target poston probablty, samplng footprnt, and path hstory. The condtonal detecton probablty for each leg s gven by 156 P k fnd = P k detect k 1 k k k l k k k ( ( t ), t ) ( 1 P ( μ ( t ), t ) exc exc exc l= 1 μ (5.5) exc exc exc exc and represents the probablty that a node wll detect the target whle traversng leg k. Usng the values of P k fnd, the total target detecton probablty s calculated as the cumulatve probablty of all legs, k ( ) Ptotal = 1 1 P fnd. (5.6) k Ths value represents the probablty that at least one node wll detect the target durng ts search path. It should be noted that ths s only an estmated value, based on the sensor and path evaluaton models presented n ths secton. 5.3 Prevous Approaches The ssue of creatng optmal paths for one or more agents searchng for an unknown target wthn an area of nterest has been a subject of research for many years. However, all these approaches vary n ther defnton of the problem from that presented n Sectons 5.1 and 5.. One of the most common approaches s to dscretze the movement of the target and agent(s) to a grd of possble postons. The movement of the target s often modeled by a stochastc process, defnng the probablty of the target randomly movng from one grd locaton to another, begnnng at a known start locaton. A search path then conssts of a seres of grd locatons, often subject to a maxmum number of steps. In [64],

169 157 solutons for ths problem model are compared for random, serpentne, and myopc search strateges for one and two agents. In [65], the authors compare seven soluton approaches ncludng Branch and Bound, Local Search, and Genetc Algorthm for one, two, and three agents. The approach presented n ths chapter dffers from these works n that an estmate of the target velocty s known, allowng the searchng agents to be better drected. In addton, the approach n ths chapter results n search paths defned over a contnuous, rather than dscrete space. The authors of [66] present a search path plannng method for the creaton of a contnuous search path for a sngle acoustc agent. Search paths have fxed leg length wth the turn angles between legs treated as soluton parameters. Solutons are generated usng a Genetc Algorthm, whch attempts to maxmze the cumulatve detecton probablty for multple targets. The scope of the problem s sgnfcantly reduced by the assumpton that target postons are statonary and defned by known PDFs. In the presented examples, target postons are determnstc. To the knowledge of the author, no prevous approaches nclude an estmated, uncertan, and evolvng target poston wth multple search agents n a contnuous path space. The combnaton of all these factors ncreases the complexty of the drected search problem over the problems consdered n prevous approaches. 5.4 Drected Search Soluton Approach Due to the complexty of the drected search problem, a two-tered strategy s taken n the soluton approach. The frst stage s the leg-by-leg ntalzaton. In ths stage,

170 158 an ntal soluton s generated sequentally, startng wth the frst leg of each search path, usng Dfferental Evoluton and an estmated ftness functon. Ths ntal soluton s used as a startng pont n the second stage, the global search. In the second stage, an Evolutonary Strateges algorthm s used to consder the entre soluton, wth performance evaluated usng the ftness functon from Secton Leg-by-Leg Intalzaton The leg-by-leg ntalzaton creates an ntal search path for each node gven an estmated track and a set of ntal parameters. These parameters are the ntal node locatons, x n 0, ntal tme, t 0, and maxmum node speed, v n max. The frst pont of each path s ntalzed usng these parameters, x n 1 n 1 t = x = t 0 n 0 n { 1... N } The second pont of each path s placed to acheve an approxmate near-optmal ntercept trajectory wth the evolvng target poston PDF. The poston x n ntercept gves the approxmated ntercept path of node n wth a contour of the target poston PDF. The calculaton of x n ntercept s dscussed n Appendx A.. The length of each ntercept leg s then normalzed, so that the tme steps of the ntercept legs for all nodes are the same as that of the shortest leg. Ths keeps the waypont tmes for all search paths synchronzed. The waypont tme for the second pont s defned so that each node moves at maxmum speed, as s the case wth all ponts n the search path. Accordngly, the second pont for each path s defned by DS. (5.7)

171 159. (5.8) After the frst leg, the remanng path legs are generated usng a fxed tme step nterval, t step, between wayponts. Snce waypont tmes for each leg are the same for all nodes, t n k wll be shortened to t k n ths secton. Usng the fxed tme step, the next waypont tme s gven by t k = t k-1 + t step. For each new leg k, a desred poston, y n k, for the next waypont s chosen for all nodes smultaneously. Collectvely, the desred wayponts for all nodes are referred to as 1 N Y, wth Y [ y y... y DS ] =. To evaluate the ftness of these waypont postons, an k k t 1 x n = t n = x =... = t n 1 + k N DS n n ( xntercept x1 ) ( n )( n v t ) n { 1... N } x n ntercept = mn 1 m N DS x n 1 estmated ftness functon, f est (Y), s used for each leg. In order to quckly evaluate the estmated ftness functon, a pre-generated target poston probablty map s created at each tme step. Ths probablty map s Equaton 5.5 wthout a samplng footprnt. It s evaluated over a grd of postons wth resoluton δx and s defned for leg k by P k est When evaluated, the value of y s rounded to the nearest grd locaton. In addton to estmated detecton probablty, the estmated ftness functon has three addtonal components. The second component penalzes the ftness based on the dstance between every combnaton of two ponts, y m k and y n k. Ths penalty ncreases as the ponts approach and prevents multple nodes from nvestgatng the same hgh probablty regon. The thrd component penalzes the ftness based on the dstance x max m ntercept v m max x l ( ) = Ppos ( y, tk tstep ) ( 1 Pexc ( y, tk tstep ) y. (5.9) l m 1 DS

172 160 between the desred waypont y k and prevous waypont x k. Ths encourages nodes to search nearby for hgh probablty regons rather than searchng far from ther current poston. The fnal component penalzes the ftness based on the dstance between the desred waypont for the current leg and the desred waypont of the last leg, adjusted for the expected movement of the target. Ths prevents nodes from movng on to a new regon due to a slghtly hgher target probablty, but stll allows them to move on f a sgnfcantly hgher probablty s found. Usng a weghted average of all four components, the per-leg estmated ftness s gven by f est N DS N DS N DS k n m n ( Y) = w P ( y ) w exp{ λ y y } w prob n= 1 N DS dst n= 1 y est m k k x n k 1 prox m= 1n= m+ 1 w n ( y + t u ) In order to choose a set of desred postons for each leg, a Dfferental Evoluton algorthm s used. Dfferental Evoluton (DE) s an evolutonary algorthm whch uses the dfference n values between ndvduals as ts prmary means of exploraton rather than random manpulatons of parameters, as n ES [67]. DE s a relatvely new evolutonary algorthm, created by Storn and Prce n 1994 [68], and has snce ganed popularty as a stochastc optmzaton tool. For each new leg k, a new DE search s run. The DE algorthm searches n the space of desred poston vectors, Y, for the current leg. The ntal populaton of vectors s created through random perturbatons of the current poston of each node. The ntal vector for the th ndvdual s gven by N DS des n= 1 y m k k 1 prox k step 1 N DS [ xk 1 xk 1... xk 1 ] + σ nt [ 1, N ] DS vel k. (5.30) (1) Y = NormRand, (5.31)

173 161 where NormRand s a vector of zero-mean, unt-varance normally dstrbuted random numbers of length N DS. After the ntal populaton of Pop leg vectors s created, the search runs for G leg generatons, accordng to the pseudo-code of Fgure 5-4. Intalze Populaton For g = 1 to G leg For = 1 to Pop leg End End Generate random ntegers s s, s [ 1, ] that s1 s s3 1, 3 Pop leg such ( g ) ( g ) ( g ) Generate chld vector: Y = Y + ( Y ) ( g ) If f ( Y ) > f ( Y ) est chld ( g + 1 ) Y = Y Else g+ 1) ( Y = Y End chld ( g ) est s1 s chld K Y s3 Fgure 5-4: Dfferental Evoluton pseudo-code for a sngle leg of the leg-by-leg ntalzaton The most ft vector n the fnal generaton s then dentfed and desred waypont postons for each node are extracted accordng to 1 N ( G ) DS leg [ y y... y ] argmax{ ( )} Y best = f Y = best best best est. (5.3) Usng these values, the next waypont s added to the ntal soluton for each node. Ths waypont s n the drecton of the desred waypont poston, but wth a leg length correspondng to the fxed tme step and maxmum speed, gven by y x n n n n best k 1 n x k = xk 1+ t stepv max n { 1... NDS}. n n (5.33) ybest xk 1

174 16 The ntalzaton then moves on to the next set of legs for each node and contnues untl the stoppng condton s reached. When the total search path soluton shows no mprovement n the past three legs, the process s stopped. Ths stoppng condton uses the overall soluton ftness functon and s gven by F F F DS DS DS ( Sk ) FDS ( Sk 1 ) < 0 ( Sk 1 ) FDS ( Sk ) < 0 ( S ) F ( S ) < 0 k DS where S k s the leg-by-leg soluton after the k th leg. When the stoppng condton s met, a fnal leg s added whch returns each node to ts ntal poston, k 3, (5.34) x t n k + 1 n k + 1 = x x = n 0 n k + 1 v x n max n k n { 1... N } DS. (5.35) 5.4. Global Search Once the leg-by-leg ntalzaton s complete, a global evolutonary search s run to mprove the results. The populaton of ths evolutonary search s ntalzed usng the result of the leg-by-leg ntalzaton. In each generaton of the global search, changes are made at random to all parameters of the soluton smultaneously, rather than one leg at a tme. Due to the large soluton space, a proper ntalzaton usng the leg-by-leg result s mportant n order to allow the global search to be most effectve. The global search s based on the Evolutonary Strateges (ES) algorthm, dscussed n the prevous chapter. Potental solutons n each generaton, g, exst n parent and chld populatons, P (g) and C (g). Each generaton, chld solutons are created and

175 163 compete wth each other and ther parents to survve to the next generaton. Before the ES search can begn, an ntal populaton must be generated. Ths populaton s ntalzed wth copes of the soluton from the leg-by-leg ntalzaton, S nt, contanng waypont poston and tme parameters, S nt.x n j and S nt.t n j. All members of the ntal populaton except one are subjected to normally dstrbuted random mutatons n poston parameters. One soluton s left as an exact copy of the ntal soluton. The ntalzaton process s descrbed by the pseudo-code n Fgure 5-5. The varable P (g) represents the th soluton of the parent populaton n the g th generaton. For = to Pop glob For n = 1 to N DS For j = 1 to N w n End P End End AdjustPath(P (1) ) (1) n (1) n. x j = P. x j + σ globint NormRand[1,] Fgure 5-5: Global search ntalzaton pseudo-code The functon AdjustPath(S) s used to adjust waypont postons and tmes n solutons such that a constant tme step and speed are mantaned for all nodes. The constant tme step requrement does not apply to the frst or last leg. These are the ntercept and return legs, respectvely, for each path. The process of adjustng soluton paths for constant tme steps and speeds s gven by the pseudo-code n Fgure 5-6. Paths are adjusted startng wth the frst leg. If a path leg has an ncorrect tme step or speed, the leg s scaled along ts current drecton to the correct length. Ths s accomplshed by changng the poston of the latter of the two wayponts whch form the path leg.

176 164 An example of path adjustment s shown n Fgure 5-7. In ths example, the path before adjustment s marked wth a sold lne. The waypont marked wth the sold blue crcle s then mutated to the poston marked by the sold blue square. The path adjustment procedure s then appled, resultng n the path marked wth a dashed lne and square wayponts. Ths new path mantans the drecton of the desred mutaton whle also mantanng a constant tme step and node speed for all path legs. Fgure 5-6: Pseudo-code for mantanng constant speed and tme steps Functon AdjustPath(S) For n = 1 to N DS n max n n n n v S S S t S t x x + = For j = 1 to N w n Mantan tme step: n max step n j n j n j n j n j n j v t S S S S S S x x x x x x = Mantan speed: n max n j n j n j n j v S S S t S t.x.x = + + End n max n N n N n N n N v S S S t S t n W n W n W n W = x x End End

177 n x j+3 n x j+ n x j+1 Path before adjustment Waypont to be mutated Waypont after mutaton Path after adjustment y [km] x j n 7 n x j x [km] Fgure 5-7: Path adjustment example Once the ntal populaton has been generated, the ES search begns. Each generaton, a new chld populaton s created through recombnaton. Ths chld populaton s then subjected to random mutatons n waypont poston parameters. The chld populaton s also subject to the random addton or removal of wayponts from search paths. Fnally, the selecton process determnes whch solutons survve to the next generaton. Ths process s summarzed n the pseudo-code of Fgure 5-8.

178 166 For g = 1 to G glob For = 1 to Pop glob Recombnaton: If UnfRand < precom (g) C = Recombne(P (g) ) Else (g) (g) C = P End For n = 1 to N DS Mutaton: For j = 1 to N W n If UnfRand < pmut pos Mutate(C (g).x j n ) End End Pont Addton: If UnfRand < pmut add AddPont(C (g) ) End End Pont Removal: If UnfRand < pmut remove RemovePont(C (g) ) End End AdjustPath(C (g) ) Evaluate Ftness: F DS (C (g) ) End Selecton: P (g+1) = Selecton(P (g),c (g) ) Fgure 5-8: Pseudo-code for drected search ES algorthm for global search The recombnaton process forms a chld soluton by combnng the paths of two parents, randomly selected from P (g). For each node n the soluton, a random waypont,

179 167 W, s selected. The path for ths node n the chld soluton s formed by concatenatng wayponts 1 through W from the frst parent wth wayponts (W+1) through N n W from the second parent. Ths s repeated for all nodes, n. Recombnaton s used to create a new chld soluton wth a fxed probablty, precom. If recombnaton s not used, the new chld soluton s formed by copyng the parent soluton. The mutaton process randomly manpulates the poston parameters of the wayponts n a chld soluton. It s performed wth probablty pmut pos on each waypont n the soluton, other than the frst and last. The frst and last waypont postons are fxed and cannot be mutated. The mutatons are normally dstrbuted wth standard devaton σ pos and performed accordng to C ( g ) n ( g ) n. j = C. x j + σ posnormrand[1,] x. (5.36) The pont addton process randomly nserts a new waypont nto a path n a chld soluton wth probablty pmut add. The new waypont s nserted n a random locaton, a, after the frst waypont and before the last waypont. The new waypont s postoned so that t forms an equlateral trangle wth the precedng and followng wayponts. Ths causes the newly formed legs to satsfy the fxed tme step constrant by constructon. Therefore, the nearby waypont postons wll not be dsrupted when the path s adjusted to satsfy the tme step constrant. However, all followng wayponts wll stll need ther tmes ncreased to meet the constant speed constrant. New wayponts are added to the left or rght of the current path wth a 50% probablty. The angle between the new waypont and the precedng pont s therefore defned by

180 θ new 168 ( g ) n ( g ) n π / 3 UnfRand 0.5 = { C. x a+ 1 C. xa} +. (5.37) π / 3 otherwse Usng ths angle, the poston of the new waypont to be nserted after waypont a and before waypont a+1 s defned by new ( g ) n a n max step [ cos( θ ) sn( θ )] x = C. x + v t. (5.38) The pont removal process randomly removes a waypont from a path n a chld soluton wth probablty pmut remove. The waypont to be removed from a path s randomly selected; however, the frst and last wayponts are fxed and cannot be removed. If a path has less than 5 wayponts, t s not permtted to have any ponts removed. After a waypont s removed and the path adjustment procedure s appled, the overall path length has been decreased by t step. Once recombnaton, mutaton, pont addton, and pont removal operatons have been used to form a chld soluton, t s adjusted to meet the tme step and speed constrants. Its ftness s then evaluated usng the F DS (S) ftness functon of Equaton 5.3. After the entre chld populaton has been generated and evaluated, the selecton process determnes whch solutons wll survve to the next generaton. The global ES search uses (μ+λ) selecton, choosng the new populaton from the unon of the parent and chld populatons [5]. Both eltst selecton and stochastc tournament are used. Eltsm selects the N elte solutons wth the hghest ftness score to be added to the new parent populaton for the next generaton. The remanng solutons n the new populaton are chosen usng stochastc tournament. The stochastc tournament selecton chooses N tourn solutons at random and selects the most ft soluton wth a new new

181 169 probablty of psel stoch. The resultng parent populaton s then used n the next generaton of the search. After ths process has repeated for G glob generatons, the global ES search ends and the best soluton n the fnal generaton s returned as the result of the drected search algorthm. Ths soluton contans a complete search path for each node nvolved n the search, begnnng at ts current poston and returnng to the same poston. The wayponts are chosen such that each node mantans a constant speed throughout the search path. Snce the path generaton process does not take vehcle dynamcs nto account when plannng paths, the maxmum node speed used n the drected search algorthm, v n max, should be less than the actual maxmum speed of the vehcle to allow for maneuverng. 5.5 Drected Search Results In order to examne the performance of the drected search soluton approach presented n ths chapter, the results of multple trals for varous drected search problems are presented n ths secton. Ths secton frst presents an example drected search problem, along wth the solutons generated by the leg-by-leg ntalzaton and global search steps. The effect of a change n the number of moble nodes s then examned through comparson of detecton probablty and path length. A smlar examnaton s then performed for changes n velocty uncertanty and target speed. For all trals performed n ths secton, the parameters of the drected search algorthms were set accordng to Table 5-1.

182 170 Table 5-1: Drected search parameter values Parameter Value Descrpton Problem parameters t step 0.4 hr Fxed tme step n v max 10 km/hr Max node speed r d 5.0 km Sensor radus Leg-by-leg nt. parameters δ x 0.5 km Estmated probablty map grd spacng λ prox 0.6 Exponental proxmty cost coeffcent w prob 100 Detecton probablty weght w prox 100 Node proxmty weght w dst 0.05 Leg dstance weght w des 0.05 Last desred poston weght 5.0 km Std. dev. of ntal populaton mutatons K 0.3 DE dfference weght σ nt Pop leg 150 Populaton sze G leg 100 Generatons per leg Parameter Value Descrpton Global search parameters w d 1.0 Detecton probablty weght Energy cost weght w e σ globint 0.8 km Std. dev. of ntal populaton mutatons precom 0. Recombnaton prob. pmut poston 0. Waypont poston mutaton prob. pmut add 0.05 Pont addton prob. pmut remove 0.04 Pont removal prob. psel stoch 0.6 Prob. of selectng most ft soluton 0.5 km Mutaton std. dev. σ pos N elte 15 Number of solutons selected wth eltsm N tourn 4 Number of solutons n stoch. tourn. Pop glob 50 Parent and chld populaton sze G glob 100 Number of generatons Drected Search Example Ths secton presents an example drected search problem and the solutons produced by the algorthms dscussed n ths chapter. In ths problem two moble nodes are used to search for a target wth an uncertan track. The target approaches from the north wth the respondng nodes ntally located well out of sensor range of the target. The ntal poston of the nodes and the ntal target poston PDF are shown n Fgure 5-9(a). A close up of the ntal target poston PDF s shown n Fgure 5-9(b). Ths PDF gves the ntal poston of the target to be wthn a 9.15 km area wth a 90% confdence.

183 Inset 0. y [km] P ntpos (x 0 ) 60 x 0 1 x x [km] (a) y [km] P ntpos (x 0 ) x [km] (b) Fgure 5-9: Node postons and target poston PDF for example DS problem (a) close up of poston PDF (b) 0

184 17 The target velocty PDF s shown n Fgure Ths PDF shows that the target s movng approxmately south wth an uncertan speed and that there s sgnfcant uncertanty n the target headng. The parameters whch defne the ntal poston and velocty PDFs are gven n Table v y [km/hr] P vel (v) v x [km/hr] Fgure 5-10: Target velocty PDF for example DS problem Table 5-: Track parameter values for example DS problem Parameter Value Descrpton μ ntpos [ ] km Poston PDF mean Σ ntpos Poston PDF covarance km μ vel [ ] km/hr Velocty PDF mean Σ vel Velocty PDF covarance (km/hr)

185 173 Fgure 5-11 shows the solutons generated for ths problem by the two stages of the drected search algorthm. Also shown s the target poston PDF at the ntal tme, t 0 = 0. The search paths generated by the leg-by-leg ntalzaton are marked wth dashed lnes. The ntal search paths are farly smooth between most legs due n part to the thrd and fourth terms of the estmated ftness functon. These terms encourage the nodes to explore areas whch are nearby ther current locaton and the locaton to whch they were headed n the last leg. The sharp turns n the paths are due to hgh probablty areas, whch warranted changng drectons due to ncreased value n the frst ftness term. The two ntal paths also do not overlap, due n part to the second ftness term, whch penalzes the estmated ftness when the nodes are too close together. The ntal soluton has a total detecton probablty of P total = and a total path length of E total = hr. Each search path has 1 wayponts. The search paths of the fnal soluton produced by the global search are marked wth sold lnes n Fgure The search paths are no longer as smooth, as the global search ftness functon only consders detecton probablty and path length. A number of wayponts have been removed, leavng only 10 and 7 wayponts n the paths of the frst and second nodes, respectvely. The search paths also now overlap. Ths does not result n overlappng coverage however, as the overlappng ponts are the sxth for the frst node and the thrd for the second node. There s a 63 mnute dfference n tme stamps between these two wayponts, therefore no redundant coverage results from these paths. The fnal search paths also make more sudden turns than the ntal paths, as a greater focus s placed on maxmzng detecton probablty. The fnal search paths have a total detecton probablty of P total = and a total path length of E total = 1.34 hr. The global search

186 therefore resulted n a 16.9% ncrease n total detecton probablty along wth a 9.13% reducton n path length compared to the ntal soluton Intal soluton Global search soluton 0. y [km] P est (x,0) x 0 1 x x [km] Fgure 5-11: Intal and global search solutons for DS example 0 The target poston probablty at tme t = 6 hr s shown n Fgure 5-1. Ths poston probablty ncludes exclusons for each of the legs n the search paths, expanded to the current tme 9. The regons surroundng path legs whch have recently been evaluated gve the lowest probablty of fndng the target, as t would have just been detected f t were present. Due to the degree of uncertanty n the velocty of the target, the nodes are unable to complete a search of the hghest remanng probablty area n the lower half of the area of uncertanty. However, by t = 6 ths regon has a target poston 9 Ths s Pest (x) evaluated at tme t = 6.

187 175 probablty of at ts peak. Ths means that any addtonal searchng n ths regon would yeld small mprovements n total detecton probablty whle sgnfcantly ncreasng path lengths. x y [km] 80 8 P est (x,6) x 0 1 x x [km] Fgure 5-1: Target poston probablty durng drected search 5.5. Performance Effects of Changes n Drected Search Parameters Ths secton examnes the effects of changes n drected search problem parameters on the performance of the search paths. The three parameters consdered are the number of moble nodes used, the mean velocty of the track, and the uncertanty of the target velocty. For each of these parameters, ts effects on total detecton probablty and total search path length are examned.

188 176 The frst parameter whch s consdered s the number of moble nodes used n the drected search, N DS. To examne the effects of changng the number of avalable nodes, the example problem of Secton s used. The ntal postons, x n 0, used n ths comparson are shown n Fgure For each value of N DS, the ntal node postons are evenly dstrbuted between x A 0 and x E 0, chosen accordng to Table y [km] P ntpos (x 0 ) x 0 A x 0 B x 0 C x0 D x 0 E x [km] 0 Fgure 5-13: Intal node postons for dfferent values of N DS Table 5-3: Intal node postons used for each value of N DS 1 N N DS Intal postons { x DS 0, L, x0 } C 1 { x 0 } { x A E 0,x0 } 3 { x A C E 0, x0, x0 } 4 { x, x B, x D x E } A 0 0 0, 0

189 177 For each value of N DS from 1 to 4, 100 solutons were generated. All parameters other than N DS were dentcal to the example problem. Fgure 5-14 shows the soluton ftness n blue and the algorthm run tme n red for the two ters of the soluton process. The soluton ftness and run tme are marked wth a dashed lne for the leg-by-leg ntalzaton and a sold lne for the global search. The data ponts represent the mean of 100 trals, whle the error bars show the magntude of the standard devaton. The plot shows that the algorthm runtme ncreases wth the number of moble nodes and that the global search consumes the majorty of the run tme. The plot also shows that ftness ncreases up to N DS = 3, after whch pont t decreases. Wth 4 nodes, the ncrease n detecton probablty s not suffcent to offset the ncreased energy cost Ftness (ntal) Soluton tme (ntal) Ftness (global) Soluton tme (global) Ftness (F DS ) Soluton tme [sec] N DS 0 Fgure 5-14: Soluton ftness and algorthm runtme for varyng N DS

190 178 The ftness components for the solutons produced by the leg-by-leg ntalzaton and global search are shown n Fgures 5-15 and Fgure 5-15 shows the total detecton probablty, P total, after the ntalzaton and global search stages. The plot shows that the global search results n ncreased detecton probablty over the ntal soluton for all values of N DS. The plot also shows that detecton probablty growth slows as N DS ncreases. Ths dmnshng return s due to the fact that as more searchng s done, fewer hgh probablty regons reman to search. The hgher that P total gets, the more searchng that s requred to ncrease t. Fgure 5-16 shows the total path length, E total, n hours for the solutons generated by both stages. The plot shows that, unlke detecton probablty, the total path length growth rate does not decrease as N DS ncreases. It s due to ths dfference that the mean soluton ftness decreases for N DS = After ntalzaton After global search P total N DS Fgure 5-15: Soluton total detecton probablty for varyng N DS

191 After ntalzaton After global search 5 0 E total [hr] N DS Fgure 5-16: Soluton total path length for varyng N DS The next parameter consdered s the uncertanty n the target track. Ths uncertanty s modfed by changng the target velocty covarance matrx, as t plays the largest role n track uncertanty. Ths s because the effect of the ntal poston uncertanty s constant, whle the effect of the velocty uncertanty grows quadratcally wth tme. To alter the target velocty uncertanty of the example problem, a scalng factor k vel s used. The scaled velocty covarance matrx s calculated usng ( vel ) vel scaled Σvel = k Σ, (5.39) whch scales the standard devaton n the x and y drectons lnearly n k vel. As k vel ncreases, so does the uncertanty n the target velocty. Fgure 5-17 shows the target poston PDF at tme t 0 +4 hr for k vel = 0.5, 1, and. As can be seen from ths fgure, ncreasng k vel causes the target poston PDF to spread much more quckly. Ths makes

192 the searchng task much more dffcult as a spread out poston PDF means longer search paths wth lower detecton probablty k vel = k vel = 1.00 x y [km] P pos (x,4) x 0 1 x 0 55 x 0 1 x x [km] x [km] k vel =.00 x y [km] P pos (x,4) x 0 1 x x [km] Fgure 5-17: Target poston PDF after four hours for three values of k vel Drected search solutons were generated n 50 trals for 6 dfferent values of k vel. Other than Σ vel, all other parameters were set as n the example problem. The mean values of the ftness components of the fnal solutons are shown n Fgure The plot shows