Theoretical Analysis of the Multi-agent Patrolling Problem

Transcription

1 Theoetical Analysis of the Multi-agent Patolling Poblem Yann Chevaleye Lamsade - univesity pais 9, place du Machal de Latte de Tassigny PARIS CEDEX 16 yann.chevaleye@lamsade.dauphine.f Abstact A goup of agents can be used to pefom patolling tasks in a vaiety of domains anging fom compute netwok administation to compute wagame simulations. The multiagent patolling poblem has ecently eceived gowing attention fom the multi-agent community, due to the wide ange of potential applications. Many algoithms based on eactive and cognitive achitectues have been developed, giving encouaging esults. Howeve, no theoetical analysis of this poblem has been conducted. In this pape, vaious classes of patolling stategies ae poposed and compaed. Moe pecisely, these classes ae compaed to the optimal stategy by means of a standad complexity analysis. 1. Intoduction To patol is liteally the act of walking o taveling aound an aea, at egula intevals, in ode to potect o supevise it. This task is by natue a multi-agent task and thee ae a wide vaiety of poblems that may efomulate as paticula patolling task. As a concete example, duing the development of the Atificial Intelligent component of an inteactive compute wagame, one may face the poblem of coodinating a goup of units to patol a given ough teain in ode to detect the pesence of enemies. The quality of the stategies used fo patolling may be evaluated using diffeent measues. Infomally, a good stategy is one that minimizes the time lag between two passages to the same place and fo all places. Beyond simulatos and compute games, pefoming this patolling task efficiently can be useful fo vaious application domains whee distibuted suveillance, inspection o contol ae equied. An example of such a task is the identification of objects o people in dangeous situations that should be escued by obots [3]. Recently, many diffeent achitectues of multi-agent systems have been poposed and evaluated on the patolling poblem [1], giving encouaging esults. In paticula, it was shown in [1] that vey simple stategies implemented though eactive agents with nealy no communication ability could achieve impessive esults. Also, some authos of these papes suggested that an appoach based on patitioning the teitoy such that each agent patols in its own egion could also wok well. This pape poposes a theoetical analysis of the patolling poblem addessing the following issues : do the existing algoithms geneate optimal stategies? Ae thee efficient algoithms geneating nea-optimal stategies? Ae patolling stategies based on patitioning the teitoy good? What if the agents all follow the same cicuit one behind anothe? To answe these questions, stategies ae poposed in this pape which ae close to stategies humans could build to patol ove a teitoy. They need a cetain amount of communication/synchonisation between agents. Which multi-agent achitectue should be chosen, how to implement the algoithms will not be ou pupose hee, and the eade inteested in these questions can efe to [7]. Ou pimay pupose is to undestand moe deeply the the patolling poblem. To do this study, we will use the fomal definition of the patolling poblem intoduced in [1], pesented in section 2. Afte a shot oveview of the pevious wok, a fist class of stategies efeed to as cyclic stategies is intoduced in section 4, and it is shown that thee exists an O(n 3 ) algoithm geneating such a stategy close to optimality. In section 5, stategies based on patitioning will be analyzed and compaed to the pevious. In section 6, the esults obtained in [1, 8] will be compaed to ous. Finally, section 7 daws some conclusions and indicates diections fo futue wok. Poceedings of the IEEE/WIC/ACM Intenational Confeence on Intelligent Agent Technology (IAT 04)

2 2. The patolling task Many tasks (such as escuing, tacking, detecting, etc...) can involve some sot of patolling, which may exhibit slightly diffeent chaacteistics accoding to the domain and cicumstances. It is then necessay fo ou study to have a moe pecise definition of patolling. In [1], it was shown that in many applications of the patolling poblem, the teitoy could be epesented by an undiected gaph. Given such a gaph, the patolling task efes to continuously visiting all the gaph nodes so as to minimize the time lag between two visits. The edges may have diffeent associated lengths (weights) coesponding to the eal distance between the nodes. Fom now on, the gaph epesenting the teitoy will be efeed to as G(V,E), wheev = {1...n} is the set of nodes and E V 2 the set of edges of G. Toeachedge (i, j) will coespond a weight c ij epesenting the distance between nodes. Note that the gaph will be assumed to be metic 1 The time taken by an agent to move acoss an edge (i, j) will be exactly c ij. At time 0, agents will be positioned on nodes of G. When the patolling task stats, agents will move simultaneously aound the nodes and edges of the gaph accoding to a pedetemined stategy. Definition. The stategy of an agent is a function π : N V such that π(j) is the j th node visited by the agent. A multi-agent stategy Π={π 1...π }is simply defined as a set of single-agent stategies. Knowing the stategy of agent k, we can now pedict at what time a given node will be visited. Fo example, agent k will visit node π k (0) at time 0. It will also visit node π k (j) at time equal to the weight of the path π k (0)...π k (j), whichis j 1 i=0 c π k (i)π k (i+1). Fothesake of claity, the weight of a path s 0...s m will be fom now on noted c(s 0...s m ). Also, the weight of a set of edges E will be noted c(e ). Ou main goal is to find good patolling stategies. We thus need an evaluation citeion. We will use idleness citeia intoduced in [1]. Definition. Let agents patol a gaph G accoding to a multi-agent stategy Π. Theidleness of a node i at time t is the amount of time elapsed since that node has eceived the visit of an agent. The idleness of all nodes at the beginning of the patolling task is set to 0. Thewost idleness is the biggest value of the idleness occued duing the entie patolling pocess fo all nodes. It is noted WI Π (G) o just WI Π when thee is no ambiguity. Figue 1 illustates the calculation of the idleness and wost idleness fo a single agent and a vey simple gaph. 1 gaphs in which the tiangula inequality is not violated, that is, given thee nodes i, j, k connected by edges, we have c ij + c jk c ik. Notice that stategies ae not necessaily finite ove time, and that agents can patol fo eve on the gaph. Of couse, if stategies ae tested in a simulato, the patolling task will have to be stopped at one time. Thus, the wost idleness measued duing a simulation will be an appoximation of thetuevalue. Idl(1)=0 t=0 Idl(2)=0 Idl(1)=0.9 t=0.9 Idl(1)=1.8 t=1.8 Idl(1)=0.2 t=2.3 Idl(2)=0.9 Idl(2)=0.8 Idl(2)=1.3 Figue 1. Agent patolling on a gaph made of two nodes and an edge of weight 1. Its stategy is π =1, 2, 1, 2, 1,... The idleness of the nodes ae shown at vaious moments. Hee, WI π =2. In this wok, we chose the wost idleness citeion among the vaious citeia defined in [1] mainly fo simplicity easons : compaed to the aveage idleness citeion, which is the othe main citeion defined in [1], the theoetical analysis will be easie than with ou citeion. In addition, the fome is uppe bounded by the latte. Thus, minimizing wost idleness will also lead to a small aveage idleness. 3. Pevious Wok Most wok on the patolling poblem as fomulated above has been done by Machado et al. in [1]. In thei aticle, they poposed seveal multi-agent achitectues vaying paametes such as agent type (eactive vs. cognitive), agent communication (allowed vs. fobidden), coodination scheme (cental and explicit vs. emegent), agent peception (local vs. global), decision-making (andom selection vs. goal-oiented selection). Fo each agent, the choice of the next node to visit is basically influenced by two fac- Poceedings of the IEEE/WIC/ACM Intenational Confeence on Intelligent Agent Technology (IAT 04)

3 tos: (1) node idleness, which can shaed (geneated by all agents) vs. individual (coesponding a single agent visits); (2) field of vision, which can be local (agent s neighbohood) o global (entie gaph). The expeiments they conducted showed two inteesting facts: fist, agents moving andomly achieved vey bad esults. Second, agents with absolutely no communication ability which stategies consisted in moving towads the node with the highest idleness pefomed nealy as well as the most complex algoithm they implemented. In the expeiments section, we will the compae ou appoach to the two most efficient stategies of [1] : (a) The Conscientious Reactive stategy, in which the next node an agent visits is the one with the highest individual idleness fom its local neighbohood; (b) The Cognitive Coodinated stategy, in which the next node an agent chooses to visit is the one with the highest idleness fom the whole gaph, accoding to the suggestions given by a cental coodinato agent. To each the chosen node, agents move though the shotest path leading to this node. The coodinato is esponsible fo avoiding that moe than one agent chooses the same next node.moe ecently, a new appoach based on einfocement leaning (RL) techniques was developped in [8]. In thei pape, Santana et al showed that a simple Q-leaning algoithm could be used to tain agents to solve the patolling poblem efficiently. The esults of thei algoithm will also be compaed to ous in the expeiments section. The stategies poposed in this pape ae vey diffeent fom those poposed in pevious wok. They have not been developped out of a specific achitectue, o a communication scheme between agents. In fact, the implementation issues of ou stategies will not be tackled hee. Howeve, an impotant wok has been done concening the implementation of the cyclic stategies in a multi-obot envionment [7]. The next section poposes a new class of stategies which we called cyclic stategies. 4. The cyclic stategies In this section, we will fist show how cycles and closedpaths can be used to ceate efficient single-agent patolling stategies. Then, an extension to the multi-agent case will be poposed, and the esulting stategies will be shown to be nea-optimal Patolling with a single-agent Conside a single agent patolling ove an aea. The simplest stategy which comes to mind would be to find a cycle coveing all the aea, and then to make the agent tavel aound this cycle ove and ove. Applied to ou case in Figue 2. Example of multi-agent cyclicbased stategy. Stategy of these obots ae π 1 = 2, 1, 4, 5, 6, 4, 1, 3, 2, 1, 4... and π 2 = 6, 4, 1, 3, 2, 1, 4, 5, 6, 4,... which aeas ae epesented by nodes in a gaph, the notion of cycle is too estictive. In fact, in the gaph-theoy teminology, a cycle is a path stating and ending on the same node and coveing each edge at most once. Howeve, fo some gaphs such as the one on figue 2, thee does not exist a cycle coveing all nodes. Instead of using cycles, we will have to use closed-paths, which ae paths stating and ending on the same node and coveing edges possibly moe than once. A closed-path is usually epesented by a list of nodes, beginning and ending with the same node. On figue 2 fo example, the closed-path s =1, 3, 2, 1, 4, 5, 6, 4, 1 coves all nodes, and tuning ove s indefinitely seems to be a good stategy fo a single-patolling poblem. Singleagent stategies consisting in taveling along a closed-path indefinitely will be called single-agent cyclic stategies.the stategy of agent on figue 1 is a single-agent cyclic stategy based on the closed-path 1, 2, 1. The bottom agent on the ight of figue 3 is also following a cyclic stategy based on the closed-path 4, 5, 6, 5, 4. Befoe extending this idea to the multi-agent case, let us concentate on two questions: 1) which closed-path should be chosen in a single-agent patolling poblem, and 2) ae stategies based on closed-paths optimal with a single-agent? To answe the fist question, let us notice that the time taken fo a single agent patolling aound a closed-path to visit a node twice is at most equal to the length of this closed-path. Theefo, with a single agent patolling aound a closed-path s, the wost idleness will be equal to the length of s. Finding the smallest closed-path coveing all nodes will thus esult in the best possible stategy among all single-agent cyclic stategies. Let us show how this poblem elates to the well known Taveling Salesman Poblem (TSP), The taveling salesman poblem is a combinatoial optimization poblem which was oiginally fomulated as follows: given a set of cities on a map, find the shotest cycle such that each city is visited exactly once. This poblem was soon extended to metic gaphs, and became: given Poceedings of the IEEE/WIC/ACM Intenational Confeence on Intelligent Agent Technology (IAT 04)

4 a metic gaph G(V,E) with edges weighted accoding to c ij, find the shotest closed-path such that each node is visited at most once [6]. Chistofides [5] poposed an algoithm which geneates in O(n 3 ) a cycle (o a closed-path fo the metic gaphs case) whose length is less than 3 2 times the shotest cycle (o closed-path). Fom now on, S TSP will denote the closed-path being the optimal solution to the TSP, wheeas S Ch will denote the closed-path obtained by the algoithm of Chistofides. The following holds: Theoem 1. Fo a single agent, the optimal stategy in tems of wost idleness is the cyclic-based stategy based on S TSP. We aleady knew that the cyclic stategy based on S TSP was the best possible stategy among all single-agent cyclic stategies. Note that this theoem states it is also the best stategy among all possible single-agent stategies. Dueto the lack of space, the poof of this theoem which can be found in [4] will be omitted hee. Note that this esult will only be needed in section 5 to demonstate theoem 2. An immediate coolay of this theoem is that the wost idleness of a single-agent cyclic stategy based on S Ch will be at most 3 2 times the wost idleness of the optimal stategy. In addition, the S Ch closed-path is calculable in O(n 3 ), wheeas the calculation of S TSP is NP-complete. In pactice, thee exists many efficient algoithms which can efficiently build closed-paths vey close to S TSP using heuistic algoithms Extending to multi-agent case One way to extend single-agent cyclic stategies to the multi-agent case is to aange agents on the same closedpath such that when they stat moving though that path all in the same diection, they keep an appoximately constant gap between them. This leads to the following definition: Definition. Let S = s 0...s m be a closed-path visiting all nodes of a gaph G. The stategy Π={π 1...π } is a multiagent cyclic stategy based on S iff thee exists d 1...d N such that π i (k) =s (k+di) mod m. The set of all multi-agent cyclic stategies will be efeed to as Π cyclic. Figue 2 illustates this with a 2-agent cyclic stategy based on the closed-path 2, 1, 4, 5, 6, 4, 1, 3, 2, such that d 1 =0and d 2 =4. How does the wost idleness evolve when the numbe of agents gows? The following lemma shows that if agents follow a multi-agent cyclic stategy Π and if d 1...d ae well chosen, then the wost idleness will be appoximatively times lowe than the wost idleness obtained by a single agent patolling aound the same closed-path. Lemma 1. Let S = s 0...s m be a closed-path coveing each node of G such that thee exists a node x V coveed exactly once by S. Letl = c(s) be the length of the closed-path. Thee exists a multi-agent cyclic stategy Π={π 1...π } based on this closed-path such that l max{c ij (i, j) E S } WI Π l +max{c ij (i, j) E S }.Hee,E S efes to the set of edges pesent in S. Note that l is also the wost idleness of the single-agent cyclic stategy based on S. Poof. Let S = s 0...s m, the closed-path coveing G. Conside two agents moving aound S such that at time 0, agent 1 is positioned on node s 0 and agent 2 is on node s d. Let l(i, j) = j 1 k=i c s k s k+1. Thus, l(0,d) is the time taken by agent 1 stating at s 0 to each node s d. Note that at any time t, the node visited by agent 2 will also be visited by agent 1 at time t + l(0,d). Of couse, a node visited at time t by agent 1 will be visited by agent 2 at time t + l(d, m). Theefo, we have WI {π1,π 2} max{l(0,d),l(d, m)}. In addition, if agent 2 visits node x at time t x, because node x is pesent only once in S, the next visit of the an agent to node x will occu exactly at t x + l(0,d). Thus WI {π1,π 2} =max{l(0,d),l(d, m)}. Let us genealize this to agents. We get WI Π = max{l(0,d 1 ),l(d 1,d 2 ),...,l(d 1,m)}. We will now have to choose the values of d k such that the wost idleness is as low as possible. By setting each d k to the geatest intege veifying l(0,d k ) k l(0,m),wegetd 0 =0and d = m. We can now wite WI Π =max k=0.. 1 l(d k,d k+1 ).Let us now calculate uppe and lowe bounds to l(d k,d k+1 )= l(0,d k+1 ) l(0,d k ) by showing that k l(0,m) max{c ij (i, j) E S } l(0,d k ) k l(0,m), thus (k +1) l(0,m) max{c ij (i, j) E S } l(0,d k+1 ) (k +1) l(0,m). Combining these two equations, we get fo all k : l max{c ij (i, j) E S } l(d k,d k+1 ) l +max{c ij (i, j) E S }. Because WI Π =max k=0..m 1 l(d k,d k+1 ), the lemma follows Optimality of cyclic stategies We have shown peviously that cyclic stategies based on S TSP wee optimal fo single agents. We will now show a simila esult fo the multi-agent case. Fom now on, opt will efe to the wost idleness of the optimal stategy. Theoem 2. Let G=(V,E) a connected metic gaph and let agents patolling on it. Let Π Ch be the multi-agent cyclic stategy based on S Ch. We have WI ΠCh 3 opt +4 max ij {c ij }. Note that if all edges of G have the same length, then opt max ij {c ij }, and theefo, WI ΠCh 7 opt. To pove this theoem, we fist need to demonstate the following lemma. Lemma 2. Fo any multi-agent stategy Π, thee exists multi-agent stategy Π = {π 1,..., π } such that fo each k 1.., each stategy π k consists in moving though a Poceedings of the IEEE/WIC/ACM Intenational Confeence on Intelligent Agent Technology (IAT 04)

5 path s k 1...sk m k of m k nodes fowads and backwads indefinitely, and such that WI Π 2max k c(s k 0...sk m k ) 2 WI Π. Poof. Conside the multi-agent stategy Π={π 1...π k }. Let s k 1...sk m k be the list of nodes visited by agent k accoding to stategy π k between time t = 0 and time t = WI Π. Clealy, k=1.. {s k 1,...,s k m k } = V. Let π k = sk 1...s k m k,s k m k 1,...,s k 1,... be the stategy consisting in moving fowads and backwads though s k 1...sk m k. Time taken by agent k using stategy π k to visit a node twice is at most 2 c(s k 1...sk m k ). Thus, WI Π (G) 2max k c(s k 1...sk m k ) 2 WI Π (G). Now the theoem can be poven. The peceding lemma will be used on the optimal stategy to build a set of paths, which will be combined to fom a closed-path leading to an nea optimal stategy. Poof of the theoem. Conside an optimal stategy Π. We can tell with lemma 2 that thee exists a stategy Π = {π 1,..., π } in which each stategy π k consists in moving though a path s k 1...sk m k andsuchthat WI Π 2 max k {c(s k 0...s k m k )} opt. Let us fist show that thee exists a closed path S coveing all nodes and such that c(s) 2 max k {c(s k 0...sk m k )} +max ij {c ij }. Let U be the set of edges pesent in these paths; because these paths cove all nodes of G, by adding at most 1 edges to U, we obtain a set U of edges such that the gaph (V,U ) is connected. Clealy, thee exists a tee T U coveing all nodes. In addition, c(t ) c(u ) k=1 c(sk 0...sk m k )+( 1) max ij {c ij }. Thee exists a closed-path S coveing all nodes obtained by exploing the tee T in a depth-fist manne (thus twice each edge), such that c(s) 2 = c(t ) max k {c(s k 0...s k m k )} +max ij {c ij }.Remembe that the closed-path S Ch obtained by Chistofides algoithm is at most 3 2 times longe than the shotest closedpath coveing G. Thus, we have: c(s Ch ) 3 2 c(s) 3 max k {c(s k 0...ck m k )}+3 max ij {c ij } 3 opt+ 3 max ij {c ij }. With lemma 1, we can geneate fom S Ch a multiagent cyclic-based stategy Π Ch such that WI ΠCh c(s Ch ) +max ij {c ij } 3.opt +4. max ij {c ij }. It is clea that because of the max ij {c ij } in the theoem, cyclic stategies will not be suited fo gaphs containing long edges. Fo this eason, let us study anothe kind of stategies, which we will call patition-based stategies. 5. Patition-based stategies Anothe vey intuitive way to make agents patol ove a teitoy would be make a patition of this teitoy into disjoint egions, and to have each agent patolling inside a single egion Figue 3. On the left: cyclic stategy Π cyc = {π cyc1,π cyc2 } and to the ight: patition based stategy Π P = {π P1,π P2 } with two agents. We have π cyc1 = 1, 2, 3, 4, 5, 6, 1, 2, 3... and π cyc2 = 4, 5, 6, 1, 2, 3, 4, 5, 6, 1... Also, π P1 = 1, 2, 3, 2, 1, 2,... and π P2 = 4, 5, 6, 5, 4, 5,... Thus, WI Πcyc =3and WI ΠP =4. Fom now on P = {P 1...P } will denote a patition of V. Thus, P 1... P = V and P i P j =. Also, {G 1...G } will efe to the subgaphs induced by the patition. Thus, G i =(V P i,e (P i P i )). Definition. A multi-agent stategy Π = {π 1...π } is said to be based on a patition P iff each agent k following stategy π k visits the nodes of a single egion of P.The class of all stategies based on patition P is efeed to as Π P. The pevious definition does not specify what moves agents should make in thei own egion. Given a patition P, how should agents behave inside thei egion? The following lemma bings an answe. Lemma 4. Fo k 1..,letπ k be the single-agent cyclic stategy based on the TSP of G k. Then, Π={π 1...π } is the optimal stategy based on patition P. Poof. Fo any stategy Π={π 1...π } based on patition P, each node of G will not be visited by moe than one agent. Thus, WI Π (G) =max k {WI πk (G k )}. Theefo,a set of optimal single-agent stategies in P 1...P is an optimal multi-agent patition-based stategy. Theoem 1 stated that single-agent cyclic stategies based on TSP wee optimal. Thus, by combining such single-agent stategies, we will obtain an optimal patition-based stategy Compaing cyclic and patition-based stategies Figue 3 illustates how both stategies pefom on a cicula gaph with two agents. On this example, the cyclic stategy wins. Conside now figue 2. On this figue, if the distance between node 1 and 4 was huge, the cyclic stategy would be disastous, as both agents would spend most Poceedings of the IEEE/WIC/ACM Intenational Confeence on Intelligent Agent Technology (IAT 04)

6 time cossing the edge (1, 4). Howeve, a stategy based on the patition {{1, 2, 3}, {4, 5, 6}} would not have this poblem. Thus, it seems that on gaphs having long coidos connecting two sub-gaphs as on figue 2, patition-based stategies could be bette. Most of the poofs will be omitted in this section, due to lack of space. Howeve, the complete poofs can be found in [4]. The following theoem compaes the values of the wost idleness of the optimal cyclic stategy and the optimal patition-based stategy. Hee opt ΠCycle and opt ΠP efe to the wost idleness of the optimal cyclic stategy and of the stategies based on patition P. Theoem 3. opt Πcycle opt ΠP +3 max ij {c ij } sketch of the poof. The pevious lemma showed that the optimal patition-based stategy was composed of single-agent cyclic stategies based on TSP. Let STSP k denote the closed-path being the solution to the TSP on subgaph G k. By joining togethe these closed-paths, it is possible to build a closed-path S coveing all nodes such that c(s) k=1 c(sk TSP )+2 max ij{c ij }. Theefo, we have c(s) k=1 c(sk TSP )+2 max ij{c ij } max k 1.. {c(stsp k )} +2 max ij{c ij } opt ΠP +2 max ij {c ij }. Fom closed-path S, we can build a multi-agent cyclic stategy Π S using lemma 1. WI ΠS c(s) +max ij {c ij } opt ΠP +3 max ij {c ij }.Thus,given any patition P, thee exists a cyclic stategy Π S veifying the pevious equation. To conclude this section, we can say that cyclic stategies ae to be pefeed when gaphs do not have long edges connecting fa egions. Othewise, building a patition of the gaph and making agents follow cyclic stategies based on the TSP of the egions is a good solution. The following coolay is a computable vesion of the main theoem. Coolay. Let P = {P 1,..., P } a patition of V.Itis possible to compute in O(n 3 ) a cyclic stategy Π Ch such that WI ΠCh 3 2 opt Π P +4 max ij {c ij }. 6. Expeiments Six diffeent gaphs (fig 4) wee poposed in [9] as a benchmak fo the patolling poblem. To evaluate the cyclic stategy on these gaphs, the TSP of each gaph was computed using the open-souce softwae Concode 2 which contains efficient heuistic algoithms which ae in pactice much faste than the O(n 3 ) Chistofides algoithm [5], and often come much close to the optimal. Then, the simulato descibed in [1] was used to measue the idleness of agents patolling aound the obtained closed-paths. The gaphs on figue 5 show the pefomance of the stategies descibed in [1] and in [8] on the six gaphs fo 2 available at Figue 4. Gaphs used duing expeiments 5 and 15 agents pe gaph. On each gaph, the cyclic stategy obtains the best esults. When the numbe of agents incease lagely above 15, all stategies become equivalent. 7. Conclusion We have shown vaious theoetical esults fo the patolling poblem. Fist, we have shown that the single-agent patolling poblem could be solved with a TSP appoach. Then, we defined the class of cyclic stategies, based on an extension of this appoach to moe than one agent. An appoximation esult was obtained fo this class, showing that in O(n 3 ), a close to optimal stategy could be obtained. The stategies based on a patitioning of the gaph wee also studied. A supising esult was obtained : except when max ij {c ij } is big, the cyclic stategies based on TSP wee shown to be bette than any patition based stategy. Howeve, when gaphs have long tunnels sepaating egions, the cyclic stategies ae not well suited. Finally, some expeiments wee conducted to compae the state-of-the-at patolling algoithms to the cyclic stategy based on TSP. The esults ae encouaging, but would need to be pusued on moe gaphs. Many othe inteesting theoetical esults have been obtained and will be published soon, in paticula concening othe kind of stategies offeing a good compomise between cyclic stategies and patition based stategies. The cyclic stategy based on TSP have aleady been to implemented in a multi-obot patolling poblem with Poceedings of the IEEE/WIC/ACM Intenational Confeence on Intelligent Agent Technology (IAT 04)

7 Figue 5. Idleness on diffeent gaphs with 5 agents (left) and 15 agents (ight). The stategies compaed hee ae the Conscientious Reactive (CR), the Cognitive Coodinated (CC), the einfocement leaning (RL) and the cyclic stategy (CS) a noisy envionment [7]. Many modifications had to be bought to the algoithm in ode to synchonize de obots. It was shown that the cyclic stategy still pefoms best when the noise level is low. Refeences [1] Aydano Machado, Gebe Ramalho, Jean-Daniel Zucke, Alexis Dogoul (2002). Multi-Agent Patolling: an Empiical Analysis of Altenative Achitectues. Thid Intenational Wokshop on Multi-Agent Based Simulation. [2] Pottinge, Dave C.: Coodinated Unit Movement. Game Develope (Januay 1999) [3] RoboCup Rescue home page: [4] Chevaleye, Y. (2003). The patolling poblem. Tech. Rep. of Univ. Pais 9. Available in fench at chevaley/patol [5] Chistofides, N. (1976). Wost-case analysis of a new heuistic fo the tavelling salesman poblem. Tech. Rep. CS-93-13, Canegie Mellon Univesity, Gaduate School of Industial Administation. [6] Reinelt G., (1994), The Taveling Salesman: Computational Solutions fo TSP Applications, Lectues Notes in Compute Science 840, Spinge-Velag. [7] Sempe F., (2004), Self-oganizing goups of obots : application to patolling tasks in noisy envionments, Ph.D thesis at univesity of Pais 6. [8] Santana H., Couble V., Ratitch B., (2004), Multi-Agent Patolling with Reinfocement Leaning, Autonomous Agents and Multi-Agent Systems. (to appea). [9] Almeida A., Ramalho G., Santana H., Tedesco P., Menezes T., Couble V., Chevaleye Y. (2004), Recent Advances on Multi-Agent Patolling, SBIA (to appea). Poceedings of the IEEE/WIC/ACM Intenational Confeence on Intelligent Agent Technology (IAT 04)