Ctaton: M. Aranda, G. López-Ncolás, C. Sagüés, Y. Mezouar. Formaton control of moble robots usng multple aeral cameras. IEEE Transactons on Robotcs, vol. 3, no. 4, pp. 64-7, 5 Formaton Control of Moble Robots usng Multple Aeral Cameras Mguel Aranda, Member, IEEE, Gonzalo López-Ncolás, Senor Member, IEEE, Carlos Sagüés, Senor Member, IEEE, and Youcef Mezouar Abstract Ths paper descrbes a new vson-based control method to drve a set of robots movng on the ground plane to a desred formaton. As the man contrbuton, we propose to use multple camera-equpped Unmanned Aeral Vehcles (UAVs) as control unts. Each camera vews, and s used to control, a subset of the ground team. Thus, the method s partally dstrbuted, combnng the smplcty of centralzed schemes wth the scalablty and robustness of dstrbuted strateges. Relyng on a homography computed for each UAV-mounted camera, our approach s purely mage-based and has low computatonal cost. In the control strategy we propose, f a robot s seen by multple cameras, t computes ts moton by combnng the commands t receves. Then, f the ntersectons between the sets of robots vewed by the dfferent cameras satsfy certan condtons, we formally guarantee the stablzaton of the formaton, consderng uncycle robots. We also propose a dstrbuted algorthm to control the camera motons that preserves these requred overlaps, usng communcatons. The effectveness of the presented control scheme s llustrated va smulatons and experments wth real robots. I. INTRODUCTION Multagent systems are very nterestng n robotcs due to ther ablty to perform complex tasks wth great effcency and relablty. In ths context, we address n ths paper the problem of brngng a set of ground moble robots to a desred geometrc confguraton, whch s also referred to as formaton stablzaton. Typcally, the formaton to be stablzed s defned n terms of absolute postons for the robots to reach [], [], or as relatve poston vectors or dstances between the agents [3]. Between the two latter varetes, dstancebased formaton control [4], [5] employs smpler nformaton and does not requre a global reference frame for the robots, whle relatve poston-based methods [6], [7] exhbt stronger stablty propertes. In ths and other related multrobot problems [8], [9], dstrbuted control strateges tend to be preferred, for robustness and scalablty. A very relevant characterstc of the method we present s the use of vson. Cameras are powerful and affordable sensors that have been, and contnue to be, extensvely used for control tasks []. In the feld of multrobot systems, [] s an early example of a vson-based framework to control a formaton, whle [], [3] tackle dstrbuted moton coordnaton tasks. [4] consders vsblty constrants and envronmental obstacles n leader-follower formatons. In these works, the cameras are carred by the robots, whereas the work [5], whose core dea s more closely related to our method, employs a camera-equpped aeral robot as a supervsory unt. Ths unt s used to compute the absolute localzaton of an ensemble of ground robots and to control the members to form a grossly-modeled shape bounded by an ellpse. Multple-vew geometrc models have proven valuable to ncrease the robustness of performance n control schemes, and have been appled to multrobot control scenaros. In ths respect, the work [6], whch s at the orgn of the one we present, proposes a homography-based centralzed system wth a sngle aeral camera. M. Aranda, G. López-Ncolás and C. Sagüés are wth Insttuto de Investgacón en Ingenería de Aragón, Unversdad de Zaragoza, Span. {marandac, gonlopez, csagues}@unzar.es Y. Mezouar s wth Clermont Unversté, IFMA, Insttut Pascal, Clermont- Ferrand, France. youcef.mezouar@fma.fr Ths work was supported by Mnstero de Cenca e Innovacón/European Unon (project DPI-3), Mnstero de Educacón under FPU grant AP9-343, RobotEx Equpment of Excellence (ANR--EQPX-44) and the LabEx IMobS3 (ANR--LABX-6-). Here, multple aeral cameras are employed. Usng a current mage of a subset of the ground robots and a reference one, each camera computes a transformaton that creates a set of desred mage ponts, from whch t defnes desred moton objectves, whch are transmtted to the robots. Then, each robot computes ts actual control nput usng ths nformaton, receved from one or multple sources. Thus, our system s not centralzed (snce the control commands are generated from partal nformaton) and does not use a global reference frame. Formaton control s challengng under these condtons [4], [5]. We propose a strategy for the ntegraton of commands and we defne constrants for the ntersectons between the sets of robots vewed by the cameras that guarantee the stablzaton of the global formaton. In our method, the vsual nformaton s obtaned by aeral cameras carred by UAVs actng as control unts, whose moton needs to be controlled to ensure that the vsblty of the ground robots s mantaned. Ths has elements n common wth works on vsual coverage, such as [7], whch presents a dstrbuted algorthm for postonng multple cameras over a planar area to be covered. The work [8] studes the problem of provdng protecton to a ground convoy wth a set of UAVs, consderng the ground vehcles are statonary or move along straght lnes. In our case, the moton of the UAVs must preserve the coverage of the robots (.e. of a dynamc envronment) and s further constraned due to the need to mantan robot ntersectons. It s challengng to address ths problem from the standpont of optmalty. We focus here on effectveness, and propose a dstrbuted algorthm that ensures the requrements are met. The use of external cameras for the control task we address has the advantage of allowng the robots to carry smpler onboard equpment, as they do not need to handle costly sensng and computaton. In addton, they do not have to transmt nformaton, whch typcally consumes a lot of power. In partcular, the man contrbuton of ths paper s the use of multple cameras, whch mproves the maxmum coverage, robustness and scalablty wth respect to a sngle-camera setup. Our method s partally dstrbuted, preservng some propertes of centralzed systems (e.g. more effcent performance than dstrbuted controllers). The system can be flexbly dmensoned by selectng the approprate number of cameras for a gven number of robots and sze of the workspace. Our approach s mage-based and, therefore, does not use range nformaton, contrary to poston-based or dstance-based formaton control technques. In addton, we do not need a global reference frame for the robots (whch typcally requres addtonal sensors). All the nformaton s measured n the mages, the cameras do not share a common coordnate reference, and ther moton does not affect the control performance. A prelmnary verson of the method proposed n ths paper was presented n [9]. Wth respect to ths prevous work, we provde here several new contrbutons: ) A more precse defnton of the control method when multple cameras are used. We now dscuss specfcally aspects of the control unts that carry the cameras (UAVs), and defne the nteractons (establshed va communcatons) that need to exst between them to guarantee a correct performance of the global system. ) A formal stablty analyss of the mult-camera control method. 3) In addton to controllng the moton of the robots, here we also address the control of the moton of the cameras. We propose an algorthm that ensures that they mantan the requred ntersectons between ther sets of vewed ground robots and thus the desred multrobot task s successfully carred out. 4) The valdaton of the method va realstc smulatons and experments wth multple cameras and real moble robots.
Fg.. Overvew of the proposed multrobot control system. Fg.. Overvew of the vson-based framework mplemented by each of the control unts usng ts assocated camera. The computed desred mage postons p d for the robots are such that the sum of ther squared dstances to the current postons p s mnmum, as explaned n the text. The magebased control scheme we propose s based on drvng p to p d. II. IMAGE-BASED FORMATION CONTROL FRAMEWORK Ths secton descrbes the framework the UAV unts use to control the moton of the ground robots, relyng on mage nformaton. Let us defne next the elements of our system, whch s llustrated n Fg.. We consder a set, S, of n uncycle robots and a set of m UAV control unts, whose moton s modeled by sngle-ntegrator knematcs. Each unt carres a camera that vews a subset of robots S j S, j =,...,m. We denote N j = card(s j). From the captured mages, and usng the proposed homography-based approach defned later on n ths secton, each unt obtans the moton commands for the robots wthn ts camera s feld of vew. These take the form of desred moton vectors, and are transmtted to the robots. The robots that are vewed by multple cameras combne the multple receved commands to obtan ther moton nput, as descrbed n Secton III. The UAVs also communcate among themselves n order to control ther own moton (Secton IV), wth the goal of ensurng approprate coverage of the robotc team and the successful completon of the control task, whch conssts n makng the ground robots postons form a desred shape, up to translaton and rotaton. Let us focus next on a gven unt j. We use normalzed homogeneous mage coordnates. Our formaton control strategy uses two perspectve mages: The reference mage, whch s fxed and represents the desred confguraton as vewed by a downward-facng camera. Ths can be an actual mage captured whle the robots were n that confguraton, or a vrtual mage generated usng the geometrc constrants that defne the formaton, expressed n the mage. Each robot S j s represented by a pont p, n pxel coordnates, n the reference mage. All unts have the same reference mage of the full formaton, but only use the ponts correspondng to robots they see. The current mage, whch represents the current confguraton of the robots seen by the camera. Ths mage s requred to satsfy the followng hypotheses: the mage plane s parallel to the ground plane, and the dstances, n pxels, n the current and reference mages have the same scale. In the current mage, each robot S j s represented by a pont, p j, n pxel coordnates. If the two mages are taken wth the same camera and the moton of the sensor s planar, the two hypotheses are automatcally satsfed. Let us stress, however, that the actual moton of the perspectve camera n our method can be arbtrary. All that s requred s a way to transform each mage captured by the camera to an equvalent current mage satsfyng the assumptons above. In our prevous work [6] we descrbed n detal how ths can be acheved through a purely mage-based procedure that rectfes the captured mages usng a concatenaton of transformatons. Ths method compensates any 3D translatonal and rotatonal moton of the camera, and any scale dfferences between the reference and captured mages. In our scheme, each camera uses the ponts n the reference and current mages to compute a D rgd transformaton composed of translaton and rotaton. Specfcally, consderng S j, when ths transformaton s appled to the reference ponts p, these are converted to a set of desred ponts n the current mage, p d,j, such that the sum of squared dstances between p j and pd,j s mnmum. The soluton to ths type of rgd shape transformaton problem [] requres the coordnates of the ponts to be frstly translated so that they are centered on ther centrods. Such centerng results n two new sets of ponts p j c and p c, S j. Then, a least-squares D rotaton between them s obtaned. As shown n [9], t s possble to express t n terms of lnear mage transformatons, by computng from the two sets a smlarty parameterzed n the followng way: H j l s j cosθ j s j snθ j s j snθ j s j cosθ j. () Then, the soluton s gven by the matrx H d,j = H j l dag(/s j,/s j,), whch s a constraned homography expressng a pure D rotaton. The desred ponts n the current mage of the camera can thus be defned, for S j, as: p d,j = H d,j p c +c j p, () where by summng the vector c j p (the centrod of the current ponts) we undo the prevous centerng. Fgure llustrates the geometry behnd the process of determnng the desred mage ponts. Consderng the framework descrbed, p j = pd,j S j clearly mples the robots n ths set are n the desred formaton. Then, our control strategy reles on makng them move to satsfy ths condton. Our choce of a least-squares soluton provdes effcent task completon, snce t mnmzes the sum of squared dstances from the robots current postons to ther destnatons. Note the dffculty comng from the fact that each camera handles a partal set of robots. We need to defne the ntersectons between these sets and the nteractons between commands ssued by dfferent control unts n a way that allows the global formaton to be acheved. Ths s addressed n the followng secton. III. COORDINATED CONTROL OF THE GROUND ROBOTS In ths secton, we employ the mage-based framework presented n the prevous secton to propose a coordnated control scheme of the multrobot system employng multple cameras. We explan next how a gven control unt j computes the nformaton to be sent to a
3 requred, the uncycle robots can acheve any desred headng by rotatng n place once they have reached ther fnal postons. Fg. 3. Left: geometrc varables and control vector computed for robot by unt j, defned n ts mage. Rght-top: representaton of s global moton vector computed from mage nformaton receved from two unts j and j. Rght-bottom: state of the robot on the ground plane. gven vewed robot S j. The parameters nvolved are llustrated n Fg. 3. Usng the strategy descrbed n the prevous secton, the unt can defne p d,j n the current mage frame. The mage dstance between the current and desred ponts can be computed as: ρ j m = pj pd,j = (p j x pd,j x ) +(p j y pd,j y ). (3) We express all angles n ( π,π]. The angle ψ j m can be obtaned as: ( ) ψ j m = atan (p j x pd,j x ),(pj y pd,j y ), (4) whle φ j m s calculated drectly from the mage of the robot. The algnment error n the mage, α j m, s obtaned as αj m = φj m ψj m. Fromρ j m andαj m, the unt can express pd,j and the resultng moton vector M j = pd,j p j, n the robot s frame. Then, ths vector s what the unt transmts to the robot. Robot may receve ths control nformaton from multple unts at any tme. When ths s the case, t sums the ndvdual vectors to obtan ts global moton vector: M g = j C M j, where C s the set of ndexes of the control unts that send control data to robot. Ths strategy to ntegrate the nformaton seems ntutvely reasonable snce the robot s moton averages the motons commanded by the unts. Referrng to Fg. 3, the robot can then drectly compute the parameters α m and ρ m. For the purposes of our control strategy, we can smply express the state of the robot va the poston on the ground assocated wth the endpont of ts global moton vector. We can express ths poston relatve to the robot by the varables α and ρ. Note that α = α m and ρ s proportonal to ρ m. Ths permts us to control the robot usng the quanttes measured n the mage. In partcular, we consder the robots have uncycle knematcs. Then, the control law we propose for robot s: { v = k v sgn(cosα m)ρ m ω = k ω (α d α m), (5) where ω s defned counterclockwse, k v > and k ω > are control gans, and we defne: α d = { f αm π π f α m > π Wth ths control, each robot can travel forwards or backwards dependng on ts locaton relatve to ts moton vector. Observe that f cosα m = for some robot, the agent does not translate (snce v = ), and can only rotate n place. Note as well that the fnal headng of each robot s not controlled to a partcular value. If. A. Stablty analyss When consderng a sngle camera, the proposed homographybased control framework s asymptotcally stable, wth locally exponental convergence, as analyzed n [6], [9]. Partcularly, n [9] and n the present work we use a purely rotatonal transformaton matrx (H d,j ) not encodng any translaton, whch smplfes the stablty analyss wth respect to the system n [6]. We focus here on the mult-camera control law, and analyze ts stablty next. Note that, clearly, the desred ponts obtaned as descrbed n Secton II are ndependent of the mage frame used to compute them. Thus, for our analyss we consder henceforth that all the mage enttes used by the dfferent cameras are expressed n a common mage frame. Let us defne an undrected graph G c = (V c,e c) modelng the ntersectons between the sets S j, j =,...,m, whch are assumed fxed. In partcular, every node n V c corresponds wth one camera, and there s a lnk n E c between two nodes j and k when t holds that card(s j Sk ). We assume the graph G c s connected, and every robot s vewed by at least one camera,.e. j=,...,msj = S. In addton, we further assume the transformatons H d,j for j =,...,m are non-degenerate. Then, we obtan the followng result: Proposton : The multrobot system under the control law (5) wth multple cameras s locally stable wth respect to the desred formaton. Proof: We wll use Lyapunov analyss to demonstrate the stablty of the system. Let us defne the followng cost functon for every control unt j: V j = S j p d,j p = S j H d,j p c +c j p p, (6) where () has been used. Note that H d,j s the D rotatonal homography computed wth the robots n S j, and c j p = ( S j p )/N j. Then, we defne the followng canddate Lyapunov functon: V = j=,...,m V j. (7) Note that V s postve sem-defnte and radally unbounded. Furthermore, t can be easly seen that, thanks to G c (whch lnks the cameras that share at least two robots) beng connected, V = occurs f and only f the n robots are n the desred confguraton. We address next the study of the dynamcs of V. From (6) and (7), we can wrte: V = ( ) V j T ṗ. (8) p j=,...,m Sj Observe that from (6), for any S j we can wrte: V j = V j H d,j + p H d,j p N j k S j (p d,j k p k)+(p p d,j ), (9) consderng the non-degenerate least-squares rotaton H d,j s a dfferentable functon of the mage ponts []. Snce ths homography s computed n our method so as to mnmze the sum of squared V j H d,j s dstances expressed n the cost functon V j, we have that null. In addton, as can be deduced from Secton II, the centrods of the current and desred ponts n S j concde. Therefore, the second addend on the rght-hand sde of (9) s also null and we get: V j p = p p d,j. We can then express (8) as follows:
4 V = j=...m (p p d,j ) T ṗ = S j =...n j C (p p d,j ) T ṗ. () Robot s global vector (Secton III) n the common frame s: M g = M j = (p d,j p ). () j C j C Let us consder frst that the robots are holonomc (.e. they can travel n any spatal drecton n the plane). If that were the case, robot would move n the drecton of the vector, and thus the dynamcs of ts assocated mage pont would be: ṗ = k vm g for some k v >. Therefore, from () and (): V = k v =,...,n M g. () For the type of robots we consder n ths work (.e. uncycles), the translaton s always n the drecton of the robot s current headng, and the magntude of the moton s proportonal to that of the global moton vector (5). The msalgnment between the mage projecton (ṗ ) of the actual translaton vector and the drecton of the global moton vector s expressed by the angle α d α m. Notce that () captures the dot product of these two vectors. We can thus wrte: V = k v =,...,n M g cos(α d α m). (3) Observe that α d α m π/ and, therefore, t holds as well that V for uncycle knematcs. Then, by vrtue of the global nvarant set theorem, we can conclude that the system converges asymptotcally to the largest nvarant set n the set R = {p, =,...,n V = }, and the system s locally stable wth respect to the desred equlbrum V =,.e. wth respect to the desred formaton. Corollary : Assume that G c s a tree, card(s Sj) = f {,j} E c, otherwse, and S Sj Sk = for unequal,j,k. Then, the system converges globally to the desred formaton. Proof: We want to determne the condtons under whch V = can hold. Due to the uncycle knematcs, V can be zero when, wth no robot translatng, at least one of them satsfes cos(α d α m) = whle M g > (3). From (5), these robots wll rotate n place at that nstant, makng V <. Thus, we only need to study the case where V = due to M g =. Assume ths scenaro, whch mples all the robots are statc, holds at some nstant. Consder a leaf node j n G c connected wth a node j, and denote S j Sj = {, }. Notce that snce the N j robots controlled only by j are statc, they already are n ther desred postons computed by j, and the only possble nonzero moton vectors computed by ths camera are for robots and. The centrods of the current and desred ponts for any control unt concde (Secton II), and therefore M j = M j. Gven that and are also statc (.e. M g = for them) we also have M j = M j for =,. We consder now an orthogonalty condton for the least-squares rgd shape algnment problem we treat []. In partcular, let us assume, wthout loss of generalty, that the mage ponts are expressed n a coordnate frame such that the centrod of the ponts n camera j s null and H d,j = I 3. Then, t holds that =,...,N j (p d,j ) T p =, where ndcates a rotaton of π/ radans. As the current and desred postons concde for all robots n S j except and, t s straghtforward to see from the above expressons that (p d,j p d,j ) T (p p ) =. Together wth the prevously establshed condtons, ths clearly Fg. 4. Image ponts and moton vectors for two robots and controlled by two control unts j and j under the condtons of Corollary. mples that all moton vectors for the two robots from the two cameras must le along the lne that jons p and p. Fgure 4 llustrates the mage ponts and moton vectors resultng from all the descrbed constrants. Assume now, wthout loss of generalty, that p d,j p d,j > p p. Ths mples p d,j p d,j < p p. However, snce the dstance between desred ponts must clearly be equal for j and j, we conclude that M j = for =, and j = j,j. Then, usng that the ntersectons between S j sets contan two robots and are mutually dsjont, t s straghtforward to propagate ths reasonng from the leaf nodes through the full tree graph and obtan that M j =,j,.e. V =. Thus, V = V =, whch mples the system converges globally to the desred formaton. Remark : Global stablty guarantees such as the ones we have provded are well known to be dffcult to obtan for non-centralzed formaton stablzaton n the absence of a global reference frame [4], [5]. In our smulatons, farly dverse setups (n terms of numbers of robots and cameras, robot ntersectons, and topologes of G c) have been tested, and we have not observed local mnma (.e. the desred formaton was always acheved). Notce, n any case, that the presence of the control unts as supervsory elements provdes flexblty to escape the local mnma, n case they occurred. Ths can be acheved by swtchng to the provably globally stable system setup defned n Corollary. Observe that ths swtch can be performed smply by makng the aeral unts control only some of the robots n ther feld of vew, and does not requre the UAVs to move. Remark : Consder the case where the sets S j are tme-varyng,.e. there are ground robots enterng or leavng the cameras felds of vew durng the executon of the control. Then, when for some camera j one robot s added to S j, ths may ncrease V j nstantaneously and can cause the aggregate functon V to be non-decreasng at that nstant. Observe that these addtons wll depend on the specfc polcy used to control the moton of the cameras and regulate the changes n thes j sets. If the polcy guarantees that the number of robot addtons s fnte over tme, the graph G c wll eventually become statc and therefore the stablty of the system wll be guaranteed. Remark 3: As mentoned before the statement of Proposton, we consder the rgd transformatons used n our method do not suffer from degeneraces, whch can appear e.g. f multple robots occupy the same poston, and ther mage projectons concde. The confguratons causng these degeneraces have measure zero,.e. wll never occur n realty. Therefore, our system s globally stable n practce, as per Corollary. We note that, theoretcally, f these confguratons were consdered, only almost global stablty would be guaranteed for the system under our control strategy. IV. CAMERA MOTION CONTROL In ths secton, we propose an algorthm to control the moton of the UAVs carryng the cameras. We assume that the cameras are downward-facng and the UAVs have sngle ntegrator knematcs. We
5 consder that two UAVs whose felds of vew ntersect can communcate. Specfcally, they exchange the nformaton of whch robots they are vewng. We also assume that the condtons dscussed n the prevous secton (.e. all the ground robots are covered, and the graph G c must be connected) hold at the start of the executon. Algorthm outlnes a polcy that effectvely guarantees these requrements are always met, thus ensurng convergence of the formaton. Algorthm Moton of the camera/control unt j ) For every control unt k that s a neghbor n the defned ntal graph G c, j determnes through communcatons the set S jk = S j Sk, and the set of ground robots whch are only vewed by ts camera, S jj. ) From ts camera s mage nformaton, j determnes, for every neghborng k, whch two robots n S jk are closest to tself: S jkc = {r jk,r jk }. Then, t determnes the robot r jf n the set S jc = k {S jkc} that s farthest away from j. 3) Unt j computes n ts camera s mage the centrod, c h, of the set of ponts S jc Sjj. 4) Unt j computes ts moton command. Its horzontal velocty s toward c h, proportonal to the dstance to that pont (measured n j s camera mages). Vertcally, t moves upwards f the dstance tor jf n the mages s greater than a predefned desred dstance (.e. when r jf s near the border of j s camera s feld of vew), and downwards f t s smaller. The vertcal velocty s proportonal to the dfference between the current and desred mage dstances for r jf. 5) Unt j executes ts own moton and transmts the control commands to the robots n S j. y (m) 5 5 5 5 Cost functon V 7 6 4 x (m) 5 5 Tme (s) y (m) 5 5 5 Homography angle (rad) 4 3 x (m).5 3 4 Tme (s) Fg. 5. Smulaton results wth forty robots and fve cameras. Top: Intal (left) and fnal (rght) confguratons, showng the robots (crcles), the cameras (squares), and the crcular footprnts assocated wth ther feld-of-vew. Bottom: cost functons for every camera, and Lyapunov functon, shown n a thcker lne (left). Angles of the D rotatonal homographes computed by every camera, n a common reference frame (rght). Ths method mantans the lnks of a fxed connected graph defned ntally, G c. The man purpose of controllng the camera heght s to ensure the necessary robots stay n the feld of vew. In addton, the UAVs wll move downwards, when possble, to get a hgher resoluton vew of the robots. A mnmum heght must be defned, for safety. The horzontal moton strategy ams at mantanng good vsblty of the robots that the camera has to preserve wthn ts feld of vew. If multple UAVs detect that they are preservng the same set of robots, all but one of them must be stopped, to avod collsons. Every ground robot wll reman covered as long as t does not leave the feld of vew of multple cameras smultaneously. Ths can be easly avoded usng safety margns and approprate selecton of the control gans for the robots and UAVs. y (m) 35 3 5 5 3 4 x (m) V. SIMULATIONS In ths secton, we descrbe smulaton results to evaluate the performance of the proposed control scheme. We present frst a smulaton carred out usng Matlab, amed at llustratng the scalablty of our method. Fve UAVs and forty uncycle robots were used, wth a rectangular grd-shaped desred confguraton. The cameras felds of vew were approxmated by crcles, and the UAV motons were controlled usng the algorthm outlned n Secton IV. A cycle graph G c was used. Notce n Fg. 5 how the target confguraton s acheved whle the cameras mantan the group coverage (as llustrated by ther sensng footprnts). As expected (Secton III-A), nstantaneous jumps n the cost functons appear when new robots enter the sets S j. Stll, ths effect does not compromse the stablty of the system. We also tested our approach usng the Cobaye software package developed by the company 4D-Vrtualz. Ths s a realstc smulator www.4d-vrtualz.com Fg. 6. Results from the realstc smulaton example. Top: Intal (left) and fnal (rght) vews of the smulated settng, wth crcles plotted around the UAVs. Bottom: Fnal mage captured by one of the cameras (left). Paths followed by the robots (rght). The cameras paths are dsplayed n thn lnes, and ther fnal postons are marked wth squares. The fnal camera footprnts are shown as dashed-lne crcles. of moble robotc systems whch ncludes the modelng of dynamcs and permts real-tme operaton. We llustrate a smulaton example n an urban scenaro wth twelve Poneer 3-AT robots and three UAVs, each carryng a downward-facng perspectve camera. The sze of the mages was 8 8, and the cameras feld-of-vew half-angle was 35. The desred confguraton for the multrobot team had the shape of a crcle. Intally, the robots had arbtrary postons and orentatons, wth the three UAVs coverng them. We defned G c to be a lnear graph. The results are llustrated n Fgs. 6 and 7. The robots converged to the desred confguraton followng farly smooth trajectores, whle the UAVs jontly mantaned vsual coverage of the
6 4.4 v (m/s) ω (rad/s) v (mm/s) 3 ω (rad/s).3... Camera heght (m) Cost functon V 6 5 4 3 8 6 4 3 4 5 6 7 8 9 Tme (s) 4 6 8 4 6 Tme (s) 3 4 5 Tme (s) Homography angle (rad) UAV speed (m/s) 3.9.8.7.6.5.4.8.7.6.5.4.3.. 3 4 5 6 7 8 9 Tme (s) 4 6 8 4 6 Tme (s) 3 4 5 6 7 8 9 Tme (s) Fg. 7. Results from the realstc smulaton. Top: Lnear (left) and angular (rght) veloctes followed by the robots. Mddle row: Evoluton of the camera heghts (left) and magntudes of the UAV veloctes (rght). Bottom: cost functons for the cameras (the curve havng greater values s the global Lyapunov functon) (left), and angles of the cameras D rotatonal homographes, expressed n a common frame (rght). team, and ther 3D postons eventually stablzed. The effects of the changes n the S j sets can be observed n the dsplayed plots. VI. EXPERIMENTS WITH REAL ROBOTS We tested our control method usng four Khepera III robots movng on a planar workspace. Dfferent FreWre cameras observng the robots were used to obtan the results we present next. Crcularcoded patterns were placed on top of the robots to allow them to be detected and dentfed n the mages. Four addtonal markers were placed on the planar workspace, for mage rectfcaton purposes. Specfcally, the ponts n the captured mages were transformed to enforce the hypotheses we requre for the current mage (Secton II). The detals of the rectfcaton procedure employed can be found n [6]. A set of mage postons obtaned wth the robots formng the desred confguraton was used as the reference mage for the homography-based control computatons. The dstance between the fxed markers n the reference mage was used to fx the scale of the current mage. Thus, no metrc or camera calbraton nformaton was employed n the experments. We descrbe the results of an experment carred out wth two cameras. One of them was hand-held durng the experment, performng a moton comprsng both translaton and rotaton. Ths camera was equpped wth a lens havng a focal length of 5 mm. The other camera had a lens of 3.6 mm focal length and was fxed over the robots workspace, facng downward. Both cameras vewed all four robots, but we defned (namng the cameras and and the robots r,r,r 3,r 4) S = {r,r,r 3} and S = {r,r 3,r 4}, so as to test the performance of the proposed dstrbuted controller. The desred confguraton was square-shaped, and the control loop ran at 5 frames/s. Fgure 8 shows the tme evoluton of the veloctes sent to the robots for ths experment, computed accordng to the Cost functon V 4 8 Tme (s).e5 e5 8e4 6e4 4e4 e4 4 8 Tme (s). 4 8 Tme (s) Homography angles (rad).5.5.5.5 4 8 Tme (s) Fg. 8. Results from the experment wth real robots. Top row: Lnear (left) and angular (rght) veloctes sent to the four robots. Bottom row: cost functons for the two cameras (the dashed lne corresponds to the movng camera), and global Lyapunov functon V (thcker lne) (left). Evoluton of the angle of the D rotatonal homography computed by the two cameras. The postve-valued curve corresponds to the movng camera (rght). method descrbed n Secton III. The evolutons of the angles of the D rotatonal homography transformatons computed by the two cameras are also shown. For the fxed camera, the computed angle eventually stablzes, whereas for the hand-held one t keeps changng over tme, due to the camera moton. The same fgure dsplays the cost functons (as defned n Secton III-A) for the two cameras, and the Lyapunov functon. They all vansh as the robots converge to the desred formaton. We show n Fg. 9 mages acqured by the cameras, overlayng the traces of the robots as the control s executed. For the fxed camera, the mage paths of the robots llustrate what ther real paths were. The effects of the moton of the hand-held camera are apparent n ts correspondng robot traces. We also show a sequence of mages from a dfferent two-camera experment, ths tme wth a T-shaped desred confguraton, to further llustrate the mage moton takng place whle the control s runnng. Supplementary llustraton of the smulaton and expermental results s provded by the vdeos that accompany the paper. VII. CONCLUSION We presented a homography-based multrobot control approach where multple UAVs carryng cameras observe and control a set of robots movng on the ground plane to brng them to a desred formaton. The proposed partally dstrbuted controller s robust and scalable wth respect to the number of robots and the sze of the formaton. Its effectveness was valdated n smulatons and experments wth real robots. REFERENCES [] M. M. Zavlanos and G. J. Pappas, Dstrbuted formaton control wth permutaton symmetres, n IEEE Conference on Decson and Control, 7, pp. 894 899. [] L. Sabattn, C. Secch, and C. Fantuzz, Arbtrarly shaped formatons of moble robots: artfcal potental felds and coordnate transformaton, Autonomous Robots, vol. 3, no. 4, pp. 385 397,. [3] M. Mesbah and M. Egerstedt, Graph theoretc methods n multagent networks. Prnceton Unversty Press,. [4] D. V. Dmarogonas and K. H. Johansson, Further results on the stablty of dstance-based mult-robot formatons, n Amercan Control Conference, 9, pp. 97 977.
7 Fg. 9. Results from real experments. The desred formaton has the shape of a square (top row) and a T (bottom row). Top row: Intal mage from the fxed camera (left). Fnal mage from the fxed camera, wth the mage traces of the robots durng the control executon overlad (center). Fnal mage, wth the robots mage traces overlad, for the movng camera (rght). Bottom row: sequence of mages from another experment. The ntal mage (left), the fnal one (rght) and two ntermedate mages from the movng camera are dsplayed. [5] J. Guo, Z. Ln, M. Cao, and G. Yan, Adaptve control schemes for moble robot formatons wth trangularsed structures, IET Control Theory & Applcatons, vol. 4, no. 9, pp. 87 87,. [6] M. J and M. Egerstedt, Dstrbuted coordnaton control of multagent systems whle preservng connectedness, IEEE Transactons on Robotcs, vol. 3, no. 4, pp. 693 73, 7. [7] D. V. Dmarogonas and K. J. Kyrakopoulos, A connecton between formaton nfeasblty and velocty algnment n knematc mult-agent systems, Automatca, vol. 44, no., pp. 648 654, 8. [8] B. S. Park, J. B. Park, and Y. H. Cho, Adaptve formaton control of electrcally drven nonholonomc moble robots wth lmted nformaton, IEEE Transactons on Systems, Man, and Cybernetcs, Part B: Cybernetcs, vol. 4, no. 4, pp. 6 75,. [9] J. Yu, S. M. LaValle, and D. Lberzon, Rendezvous wthout coordnates, IEEE Trans. Automat. Contr., vol. 57, no., pp. 4 434,. [] G. López-Ncolás and Y. Mezouar, Specal ssue on vsual control of moble robots, Robotcs and Autonomous Systems, vol. 6, no., pp. 6 6, 4. [] A. K. Das, R. Ferro, V. Kumar, J. P. Ostrowsk, J. Spletzer, and C. J. Taylor, A vson-based formaton control framework, IEEE Trans. on Robotcs and Automaton, vol. 8, no. 5, pp. 83 85,. [] R. Vdal, O. Shakerna, and S. Sastry, Followng the flock: Dstrbuted formaton control wth omndrectonal vson-based moton segmentaton and vsual servong, IEEE Robotcs and Automaton Magazne, vol., no. 4, pp. 4, 4. [3] N. Moshtagh, N. Mchael, A. Jadbabae, and K. Danlds, Vsonbased, dstrbuted control laws for moton coordnaton of nonholonomc robots, IEEE Trans. on Robotcs, vol. 5, no. 4, pp. 85 86, 9. [4] D. Panagou and V. Kumar, Cooperatve vsblty mantenance for leader-follower formatons n obstacle envronments, IEEE Transactons on Robotcs, vol. 3, no. 4, pp. 83 844, 4. [5] N. Mchael, J. Fnk, and V. Kumar, Controllng ensembles of robots va a supervsory aeral robot, Advanced Robotcs, vol., no., pp. 36 377, 8. [6] G. López-Ncolás, M. Aranda, Y. Mezouar, and C. Sagüés, Vsual control for multrobot organzed rendezvous, IEEE Transactons on Systems, Man, and Cybernetcs, Part B: Cybernetcs, vol. 4, no. 4, pp. 55 68,. [7] M. Schwager, B. Julan, M. Angermann, and D. Rus, Eyes n the sky: Decentralzed control for the deployment of robotc camera networks, Proceedngs of the IEEE, vol. 99, no. 9, pp. 54 56,. [8] X. C. Dng, A. Rahman, and M. Egerstedt, Mult-UAV convoy protecton: An optmal approach to path plannng and coordnaton, IEEE Transactons on Robotcs, vol. 6, no., pp. 56 68,. [9] M. Aranda, Y. Mezouar, G. López-Ncolás, and C. Sagüés, Partally dstrbuted multrobot control wth multple cameras, n Amercan Control Conference, 3, pp. 638 634. [] J. C. Gower and G. B. Djksterhus, Procrustes problems. Oxford Unversty Press, 4.