Evaluation of Techniques for Merging Information from Distributed Robots into a Shared World Model

Transcription

1 Master Thess Software Engneerng Thess no: MSE-2004:26 August 2004 Evaluaton of Technques for Mergng Informaton from Dstrbuted Robots nto a Shared World Model Fredrk Henrcsson Jörgen Nlsson School of Engneerng Bleknge Insttute of Technology Box 520 SE Ronneby Sweden

2 Ths thess s submtted to the School of Engneerng at Bleknge Insttute of Technology n partal fulfllment of the requrements for the degree of Master of Scence n Software Engneerng. The thess s equvalent to 20 weeks of full tme studes. Contact Informaton: Authors: Fredrk Henrcsson Address: Abraham Ahléns väg 3, KRISTIANSTAD, SWEDEN E-mal: fad@swpnet.se Jörgen Nlsson Address: Araslövsvägen 11, 29175, FÄRLÖV, SWEDEN E-mal: pt00jn@student.bth.se Unversty advsor: Stefan Johansson School of Engneerng E-mal: sja@bth.se School of Engneerng Bleknge Insttute of Technology Box 520 SE Ronneby Sweden Internet : Phone : Fax :

3 ABSTRACT In the robotcs doman today there are many uncertantes. Sensors fal to provde complete and relable nformaton of the world, a world that s dynamc wth objects movng n and out of a robot s percepton. One way to mtgate these uncertantes s to share and merge nformaton between robots. However, nformaton sharng tself brngs uncertantes and questons that must be addressed. What nformaton s useful to share? How relable s t? How often should t be shared? How should ths nformaton be ntegrated wth the nformaton already present at the recever? Ths thess s ntended to deal wth the mergng of shared nformaton from several dstrbuted robots; nformaton that s manly useful for determnng the real world postons of objects. We have dentfed some technques for mergng shared nformaton and tested these n varous dfferent stuatons n an effort to determne the most approprate one. Our tests show that sharng nformaton can be very benefcal when objects are occluded from the vew of one or more robots. The tests also show that nformaton sharng s useful for decreasng object poston uncertanty. Keywords: Robots, Informaton sharng, Informaton mergng, RoboCup.

4 CONTENTS 1 INTRODUCTION RESEARCH QUESTIONS RESEARCH METHODOLOGY OUTLINE ROBOCUP TEAM CHAOS TEAM CHAOS CHALLENGES CHALLENGES OF ROBOCUP The Blnd Dog Challenge The Unknown Landmarks Challenge SHARING INFORMATION SHARED WORLD MODEL MULTI-SENSOR INTEGRATION SYNCHRONIZATION FILTERING INFORMATION AN ARCHITECTURE FOR INFORMATION MERGING IMPLEMENTED FILTERS Exponental weghted movng average flter Least mean square flter Lnear regresson flter Probablty matrx merger Lne ntersecton merger Angle group flter ERROR SOURCES EXPERIMENTS TCC TEST Localzaton test FILTER TESTS Performance test Poston estmaton test Occluded object test Share frequency test Flter reacton test Accuracy test DISCUSSION FILTERS PROBLEMS ENCOUNTERED LESSONS LEARNED CONCLUSION AND FUTURE WORK ACKNOWLEDGEMENTS REFERENCES...40 APPENDIX A...42 APPENDIX B...43

5 1 INTRODUCTION Many robots today rely on dfferent landmarks n the world to determne ther own poston, whch drectly affects ther poston estmatons of other objects. However, snce they buld ther world models ndvdually, they only rely on the nformaton that they gather themselves, whch can be nsuffcent and unrelable. By combnng nformaton from many robots, a more precse world model, where both ther own and postons of other robots are more accurate, can be bult and msnterpretatons can be mtgated. Many consderatons have to be made when sharng nformaton, such as how often do we want to share, what data should we share and wth whom should ths data be shared. In addton to ths, we have to consder how the data should be merged. 1.1 Research questons We set out to fnd what dfferent nformaton mergng technques exsted, and by evaluatng them n dfferent stuatons we sought to fnd ther advantages and dsadvantages to ultmately be able to select the best alternatve for our stuaton. Informaton sharng s affected by several factors, and we wanted to fnd out whch factors exst and how much each factor affects nformaton sharng. 1.2 Research methodology In the ntal phase, a study of relevant lterature wll be conducted. The lterature wll be evaluated and exstng technques for mergng world model nformaton wll be nvestgated and chosen for more thorough analyss. A number of these technques wll be chosen for mplementaton and practcal experments. Fnally, the resultng test data from the experments wll be presented and analyzed. 1.3 Outlne Chapter 2 begns by gvng an ntroducton to RoboCup and the Swedsh-Spansh team Team Chaos. It also descrbes the dfferent challenges of RoboCup 2004 and our contrbutons to these challenges. Chapter 3 explans the benefts and problems of sharng nformaton, and how shared nformaton can be stored wthn a world model. It also descrbes how data from several sensors can be fused together and how nformaton from several robots can by synchronzed. Chapter 4 descrbes the concept of flterng nformaton and llustrates our archtecture for nformaton sharng. It also ntroduces dfferent flters that we wll test and dfferent error sources that exst n the robotcs doman. Chapter 5 presents a number of tests, how they were performed and an analyss of ther results. The results of the tests are further dscussed n chapter 6 together wth other dscoveres we have made durng the makng of ths thess. Chapter 7 gves a fnal concluson and some thoughts on future work. 1

6 2 ROBOCUP 2004 The RoboCup ntatve was started back n 1997 by Ktano et al. wth the followng purpose: "RoboCup s an nternatonal research and educaton ntatve. Its goal s to foster artfcal ntellgence and robotcs research by provdng a standard problem where a wde range of technologes can be examned and ntegrated." [Ktano 98, Robocup.org] The ultmate goal of RoboCup s: By the year 2050, develop a team of fully autonomous humanod robots that can wn aganst the human world soccer champon team. [Robocup.org] The dynamc envronment wth teams of multple fast-movng robots makes RoboCup an nterestng area to evaluate dfferent technologes, such as: mult-agent collaboraton, reactve behavour, learnng, context recognton, strategy acquston, real-tme reasonng, sensor-fuson and many more [Asada 99, Ktano 95]. The orgnal ntatve conssted of three compettons: the real robot competton, the software robot competton and the specal sklls competton [Ktano 95]. In the real robot competton, physcal robots whch made up teams of up to 11 robots competed wth other teams on a soccer feld, whch was about 1/20 the sze of a normal soccer feld. The software robot competton s smlar, but wthout the physcal robots. Instead, competton partcpants made player control programs that communcated wth a soccer smulaton server that acted as a game envronment. In the specal sklls competton, partcpants were expected to demonstrate some specal robot sklls, such as penalty kckng, goal savng, human-lke actons etc. The RoboCup ntatve has snce then grown to nclude conferences, workshops and educaton. The competton part has also grown larger. The real robot soccer competton has been splt up based on the physcal characterstcs of the partcpatng robots [Robocup.org]: Small-Szed Robot League. Two teams of 5 small robots play wth no onboard sensors, relyng on an overhead camera to provde them wth locatons of the ball, team mates and opponents, va wreless communcaton. The robots must ft wthn a 18cm cylnder and not be taller than 22,5cm or shorter than 15cm. Mddle-Szed Robot League. Two teams of 4 md-szed robots play wth onboard sensors. Communcaton between team mates s provded va wreless communcaton. The robots must be at least 30cm x 30cm x 40cm and at most 50cm x 50 cm x 80 cm. Four Legged Robot League. Two teams of 4 Sony AIBO robots play wth onboard sensors. Communcaton between team mates s provded va wreless communcaton. Humanod League. Humanod robots show basc sklls of soccer players, such as shootng a ball or defendng a goal. There are numerous other types of compettons n addton to the soccer dscplne, for example RoboCup Rescue and RoboCup Challenges. In RoboCup Rescue, multple robots, autonomous or tele-operated, move around n a dsaster area wth the purpose of fndng vctms and buldng a map of the area. Ultmately, the goal of RoboCup Rescue s to enable the use of autonomous robots n real dsaster areas, such as collapsed buldngs, to avod puttng rescue personnel n danger. In 2

7 addton to fndng vctms and buldng maps, the goals are that the robots should be able to ascertan the condtons of vctms, place sensors, dentfy hazards, dentfy the status of the buldng and provde structural shorng. The RoboCup Challenges part of the ntatve was started to ensure steady and focused research progress n specfc areas. The challenges change every year and are dfferent for every league. A detaled descrpton of the RoboCup 2004 Legged League Challenges can be found n secton 2.3. When we menton to the challenges of RoboCup 2004 from now on, we refer to the challenges of RoboCup 2004 Legged League. 2.1 Team Chaos Team Chaos, formerly known as "Team Sweden", s a team that partcpated n RoboCup Team Chaos conssts of the followng four stes: Örebro Unversty (Sweden), Bleknge Insttute of Technology (Sweden), Lund Unversty (Sweden) and Unversty of Murca (Span). In RoboCup 2004 Örebro and Murca were responsble for the normal RoboCup football competton whle Bleknge and Lund were responsble for the RoboCup challenges. 2.2 Team Chaos Challenges Due to the growng complexty of the exstng Team Chaos software, a decson was made to create a new soluton from scratch usng the Tekkotsu framework, whch wll be used n the challenges [Tekkotsu.org]. Tekkotsu s an open-source applcaton framework created and mantaned by Carnege Mellon Unversty, USA. The goal of Tekkotsu s to provde a structure to buld upon. Tekkotsu takes care of low level routnes whch allow programmers to focus on more advanced behavours wthout havng to frst buld the basc structure. The framework ncludes moton (walkng and head control), montor tools (vson montor, jont montor), remote control and debuggng tools. Tekkotsu s bult usng OPEN-R, whch s the standard programmng nterface for the entertanment robots from Sony [Open-r.com]. Programs are run concurrently as one or several objects and the envronment allows for communcaton between objects, however the connecton has to be defned pror to executon n a separate descrpton fle. The OPEN-R SDK allows for programmng n C++ for several Sony AIBO robot models. In fact, the model s pretty much transparent to the programmer except for when dealng wth the external sensors of the robot, whch are dfferent from robot to robot and must be addressed accordngly. The new soluton was called Team Chaos Challenges (TCC). In Fgure 1 you can see the basc archtecture. Vson s a module responsble for analysng pctures taken by the vson camera of the robot. It should recognze objects such as the ball, and estmate ts poston among other thngs. Wreless s a module used for communcaton between robots; wth ths module t s possble to send messages to a specfc robot. The Behavour module decdes what the robot should try to accomplsh usng behavours n behavour state machne. In the mddle of the archtecture the WorldState s located, whch s a shared resource wth nformaton about where objects currently are postoned, when they were seen and so on. All other modules can address and use ths resource whenever they need to. The GlobMap module s a component that runs separately and s used for self-localzaton,.e. decdes where on the feld the robot currently s located. 3

8 Fgure 1. The TCC archtecture. The modules take turns to access the world state. 2.3 Challenges of RoboCup 2004 There were three challenges n RoboCup 2004, of whch we contrbuted to two of those challenges; The Blnd Dog Challenge and the Unknown Landmarks Challenge The Blnd Dog Challenge The frst of the challenges of Robocup 2004 was a so called Open Challenge, the purpose beng to promote creatvty and allow partcpatng teams to present nterestng research wthn three mnutes tme usng standard Sony AIBO robots wthout modfcatons [Challenges 04]. The TCC group decded to present a Blnd Dog challenge Problem descrpton The Blnd Dog challenge nvolved three robots, one of them beng blnd, and a ball. The blnd robot was an older Sony AIBO robot model, an ERS-210, whle the seeng robots were of the Sony AIBO ERS-7 model. The two seeng robots were placed on the long sdes of the feld, one robot on each sde. The blnd robot and the ball were placed on dfferent sdes of the feld. The goals of the blnd dog challenge were to walk close to the ball, algn wth the goal and shoot the ball nto the goal. The purpose of the seeng robots was to tell the blnd robot where t and the ball were located on the feld Our contrbuton Due to the nablty of the blnd robot to locate tself, the seeng robots had to share relevant nformaton wth the blnd robot so t had a better dea of where t should go and what t should do. The TCC mplementaton lacked a way for other robots to measure the dstance to other robots by nterpretng mages from the camera. However, the blnd robot was clothed n easly dstngushable colours and thus the angle from the seeng robot to the blnd robot could be measured and shared. The measured angle to the blnd robot was sent to the blnd robot, and t calculated an ntersecton from the angles of the seeng robots to determne ts own global poston on the feld. The same angle-ntersecton approach was appled to the ball, and tests yelded good results where the global poston of the ball only dffered 5-15 cm from the actual real world poston. 4

9 Mergng of data from seeng robots on a blnd robot has been done before, e.g. by Stroupe et al.. In ther mplementaton, the blnd robot tracks the ball only wth merged nformaton from the seeng robots. Ther results show that the trackng ablty of the blnd robot s not substantally dmnshed [Stroupe 01] The Unknown Landmarks Challenge The second challenge we contrbuted to was called "The almost SLAM Challenge" and was meant to stmulate teams to move away from strctly defned beacons to more general localzaton solutons [Challenges 04] Problem descrpton The challenge conssted of two phases. In the frst phase the football feld looked lke t usually does except for the addton of some landmarks outsde the feld. It was not known beforehand how these landmarks would look lke or where they would be placed, however some ndcatons were gven, such as the sze of the landmarks had to be between 10 and 50 cm. In phase one, the robot had sxty seconds to explore the feld, dentfy new landmarks and estmate ther postons. In the second phase, all the normal landmarks were removed and the robot would have to use the newly dscovered landmarks to estmate ts own poston. The robot had fve pars of coordnates on the feld that t would try to walk to, whle usng the new landmarks for localzaton. The robot had two mnutes to walk to as many postons as t could and the team would receve ponts dependng on how close to the specfed coordnates the robot walked and how long tme t took Our contrbuton The real problems n ths challenge could be dvded nto two parts. Frst the robot had to be able to recognze new landmarks of a more general type than the usual landmarks. Secondly the robot must estmate the poston of the new landmarks, whch s not as trval as t frst mght sound. Normally n the TCC archtecture the Vson module would estmate the dstance to an object usng the known actual sze of the object and compare ths wth how large the object s n the pcture taken by the camera. But ths technque can not be used wth the unknown landmarks snce ther actual sze s unknown. To get a good estmaton of the unknown landmarks, the dea was to let the robot observe them from more than one poston and try to trangulate ther poston. The problem of mergng observatons of landmarks from dfferent locatons made by one robot s qute smlar to mergng observatons of an object made by many robots from dfferent locatons. Therefore we decded to create a more general nformaton sharng soluton so t could be used n ths challenge as well 5

10 3 SHARING INFORMATION What s the beneft of sharng nformaton? Informaton gathered by an ndvdual robot may quckly become naccurate and unusable, especally n dynamc envronments. By sharng nformaton between robots, t allows for a more complete and up to date mappng of the perceved world. Informaton collected from multple ponts can reduce uncertanty, mprove accuracy and ncrease tolerance of errors when estmatng locatons of observed objects, f the nformaton s processed effectvely [Ca 97, Stroupe 01]. It also helps n path plannng and obstacle avodance [Kasńsk 01]. One very good reason for sharng nformaton s that one robot that sees a certan object can dstrbute ths to the others so a robot that does not see the object wll stll know where t s. Ths s good because the robot that does not see the object wll not have to spend tme lookng for t but can nstead look n the drecton of the object untl t gets a vsual of ts own [Gutmann 00]. Informaton sharng s beng used to mprove robot coordnaton, cooperaton, mult-robot learnng [Matarc 97] and role assgnment n dynamc envronments such as RoboCup [Castelpetra 00, Spaan 02, Val 03]. Informaton sharng s also beng used n exploraton, navgaton and path plannng [Ishhara 96, Reklets 01]. When t comes to nformaton sharng you can flter or merge dstrbuted observatons on dfferent levels. The concept of flterng s descrbed n chapter 4. In Fgure 2 you can see some dfferent paths you can take when sharng observatons. When a robot dentfes an object wth ts camera, t estmates the poston of the object relatve to the robot tself usng dstance and angle to the object. Ths representaton s called egocentrc data. Ths data can be transformed to a representaton that s relatve to the global world and called global data. As you can see n the pcture, there are two man paths; ether you can share the global data or the egocentrc data. When t comes to the actual flterng, ths can also be done n dfferent ways. Ether the data can be fltered n the sendng robot before t s sent or n the recevng robot after t has been sent. In the frst case, the robot can flter ts global data or even ts egocentrc data f you want to send more stablzed data. Ths can be a very good dea snce the sendng robot most lkely would not be able to send ts observatons every frame, because of networkng and processng lmtatons. When the data has been receved by the recevng robot, t can also be fltered here n two dfferent ways. Ether all the data can be fltered together wthn the same flter or just the last data from each robot can be fltered together to get a better poston estmaton. In the frst case, the data from one of the robots mght be overrepresented f t has made more observatons than the other robots, but n the second case you mght rsk to use old data from some of the robots f t has not seen the object for a whle. 6

11 Fgure 2. The fgure shows dfferent nformaton flterng paths for two robots sharng nformaton. 3.1 Shared World Model When t comes to sharng nformaton about the surroundngs between autonomous robots you need a good way to represent the data both observed by the robot tself and other robots. Ths representaton s usually called a world model. In a collectve of autonomous robots, the world model can be stored and mantaned n several ways; n each and every robot, n some sort of dedcated central server, dstrbuted between the robots [Groen 01] or a combnaton of these [Gutmann 00]. There are dfferent ways to store nformaton about where an object s located wthn a world model. You can just store the exact coordnates of each object but ths s not always a good approach snce you often do not know the exact coordnates n a dynamc envronment. One common soluton s to represent a poston of an object as a two dmensonal Gaussan dstrbuton [Stroupe 01]. A Gaussan dstrbuton s another word for normal dstrbuton, whch s a statstcal dstrbuton wth mean and varance values for a collecton of data [Mathworld]. Stroupe et al. states that several of these dstrbutons can be merged nto an mproved object poston estmaton [Stroupe 01]. Another way to store a locaton of an object s to use a matrx or grd wth a probablty value n each cell that tells how lkely t s that the object s located n that poston [Kasńsk 01]. Ths method can handle uncertan and ambguous data n a good way, but on the other hand t requres a large amount of data to be processed. When t comes to sharng a world model between several robots you have to consder the amount of data the world model conssts of snce a robot mght have a rather lmted network resource. Consderng ths, a smaller world model representaton lke a Gaussan dstrbuton mght be preferable to the probablty matrx world model. 3.2 Mult-sensor ntegraton Mult-sensor ntegraton, or mult-sensor fuson, and dstrbuted sensng are two areas that are closely ntertwned wth nformaton sharng. Accordng to Ca et al., dstrbuted sensng centres on ntegraton of sensor nformaton from multple robots [Ca 97]. The ntegraton s sometmes done by the robots themselves [Fukuda 96], but sometmes the results are ntegrated on a connected computer whch sends the results of the ntegraton back to the robots [Detl 01]. Dstrbuted sensng can be benefcal n cases when t s more expensve to buy a 7

12 sngle mult-sensor robot than several smaller robots wth fewer sensors attached. A sngle sensng robot s lmted by tme and space, but by usng dstrbuted sensng, larger areas can be covered n less tme than would be possble by a sngle sensng robot. The concept of mult-sensor ntegraton means the ntegraton of several sensors, such as vson, laser, sonar and nfrared, wth the purpose that the dfferent sensors wll provde complementary or redundant nformaton. Integraton of redundant nformaton can reduce uncertanty and ncrease accuracy of the objects that are perceved, whle complementary nformaton s needed n those cases where the objects n the envronment can only be perceved by ntegratng nformaton from multple sensors [Luo 89]. Also, f one sensor s reportng faulty measurements, the redundant nformaton from the other sensors can compensate for ths and keep the nformaton relable. The sensors can be dstrbuted or attached to the same robot, and the ntegraton can ether be done by the robot tself or by a dedcated computatonal unt. To draw parallels to the RoboCup doman, and especally the Sony AIBO legged doman, some of the avalable sensors on the Sony ERS-7 are; a camera, an nfrared sensor attached to the head rght next to the camera and an nfrared sensor attached to the chest. Of course, the camera tself can not measure the dstance to objects, but objects can be dentfed from the mages taken and, by measurng the sze of an object n pxels, a dstance can be estmated as mentoned n secton The nfrared sensor mounted on the head measures the dstance to objects from 50 to 1500 mm. So t would appear that there s a defnte possblty to ntegrate the measurements from these two sensors. However, through testng, we dscovered that the nfrared sensor attached to the head s only relable n the nterval from 25 to 750 mm as can be seen n Dagram 1, and t requres the observed object to be somewhat rght n front of the head of the robot to work satsfactory. Ths fact, n addton to the fact that the nfrared sensor cannot dentfy objects, whch makes t hard to map an nfrared observaton and a camera observaton to the same object, makes the mergng of the two sensors qute dffcult and hardly ever useful Dstance (cm) Head angle (degrees) 25cm 50cm 75cm 100cm 125cm Dagram 1. The dagram shows the results from testng the nfrared sensor of the Sony AIBO ERS-7 robot. The sensor measures the dstance to a Sony AIBO ERS- 210 robot whch was placed so the left sde of the robot was facng the measurng robot. The dfferent lnes represent the measured dstance. In ths test, the head of the measurng robot was turnng, and the angle can be seen on the x-axs. The robot, to whch the dstance was measured, was moved between the dstances lsted on the rght sde of the dagram. 8

13 The chest sensor can measure the dstance to obstacles that are n the nterval from 0 to 500mm, but t s located n such a way that the angle ponts towards the floor. It appears that the nfrared sensors are only useful n task plannng, e.g. when avodng obstacles. Ths leaves the Sony AIBO wth only one useful physcal sensor, namely the camera, and that mght not be enough to be fully aware of ts surroundngs n the dynamc world of RoboCup. Luo and Kay pont out the need for fusng nformaton from several dfferent sensors n dynamc and real-tme envronments so that as much nformaton as possble can be gathered quckly [Luo 89]. By ncorporatng deas from dstrbuted sensng, but by makng the nformaton mergng fully dstrbuted, the Sony AIBO has the extra sensors,.e. the other robots, to keep t up to date about what s happenng n ts surroundngs. The concept of logcal sensors s somethng that unfes the treatment of dfferent sources of data n mult-sensor systems [Petru 92, Kasńsk 01]. Accordng to Kasńsk and Skrzypczyńsk, logcal sensors can be thought of as abstract nformaton sources consstng of a data source; e.g. a physcal sensor such as a camera or an nfrared sensor, but also an a pror map such as a world model of the RoboCup feld, and a data processng method. The data processng method determnes what to do wth the gathered data. To draw parallels to the TCC mplementaton; the camera s a data source that takes mages of the world, whle the data processng method s the way the mages are processed. In the TCC mplementaton, the mages are processed n several ways. Shapes and colours of known objects are dentfed n the mages. From the sze of the shapes, a dstance can usually be estmated. If the robot knows ts own locaton on the feld, t can determne the global poston of the dentfed object by calculatng the angle to the object, whch s ultmately put n a global map. The way that ths connects to nformaton sharng s that the data processng methods must be able to provde data that can be nterpreted by the robots wth whom the data s shared. 3.3 Synchronzaton When sharng nformaton n a real-tme dynamc envronment, t s essental that the recever of the nformaton knows when the receved data was gathered. Wthout some knd of tmestamp, t s qute dffcult for the recever to verfy the valdty of the nformaton, because t could just as well be very old and most lkely naccurate as very fresh and accurate. As mentoned by Roth et al., the latency n the wreless communcaton on the 2002 AIBO robots was experenced as very severe, e.g. sent messages could arrve at the recever n 0.5 seconds but just as well n 5 seconds [Roth 03]. Ths hgh latency hndered them from usng synchronzaton between robots. However, our tests have snce then shown that the communcaton protocol has mproved enough for effectve use of synchronzaton. In the tests, the latency was measured to be no more than 0.5 seconds at most. Stroupe et al. have done nformaton mergng wthout synchronzaton. They broadcast the locatons of objects every cycle and state that t takes at most 120 mllseconds for the data to arrve. Data from older cycles are dscarded [Stroupe 00]. In addton to the latency, the robots are not usually started at exactly the same tme, resultng n tme dfferences between the robots. Whenever a robot receves nformaton from another robot, t has to convert the observaton tmestamp to ts own local tme, by takng latency and tme dfference nto account. In the TCC mplementaton, each robot mantans a connecton to every other robot on ts team, sendng synchronzaton messages at constant ntervals, n ths case every thrd second. 9

14 The synchronzaton protocol works as follows: Fgure 3. The fgure shows the synchronzaton protocol n the mplemented soluton. Apart from respondng wth acknowledgements, Robot2 s dong the exact same procedure as Robot1 s dong n ths fgure. 1. Robot1 sends a message (SYNCH1) contanng ts current local tme (LTIME1) to Robot2. 2. Robot2 responds wth an acknowledgement (ACK1) contanng ts local tme (LTIME2) when SYNCH1 was receved and LTIME1. Robot1 receves ACK1. By subtractng LTIME1 from the current local tme, Robot1 calculates the latency. To calculate the tme dfference between Robot1 and Robot2, Robot1 subtracts (LTIME2 + latency/2) from ts current local tme. The calculated dfference s entered nto a mean value flter to keep the tme dfference robust. The same procedure s done concurrently by Robot2 and by any other robot partcpatng n the sharng of nformaton. The dfference value s used whenever nformaton s receved from another robot. The dfference s added or subtracted from the nformaton tmestamp, resultng n the nformaton tmestamp beng converted to the local tme of the recevng robot, and thus makng t usable for that robot. 10

15 4 FILTERING INFORMATION Wthn the sgnal processng area, a flter s an algorthm that takes some nput sgnal, processes t and produces an output sgnal. The processng part of the flter can perform nformaton mergng, whch s why we n the thess call nformaton mergers flters. There exst many dfferent methods for mergng or flterng nformaton. One method for mergng nformaton s to use a weghted average of a range of values. Luo and Kay descrbe ths method as one of the smplest and most ntutve fuson methods [Luo 89]. It s both easy to understand and mplement. A weghted average flter would take a number of sensor readngs and add them together at the same tme as they are multpled by a specfc weght. Ths weght can for example be a rate of how accurate the value s. When usng a weghted average flter on a value, t stablzes and less accurate sensor readngs wll not be trusted as much. Of course, ths presumes that you have some knd of ratng of the accuracy of the value beng fltered. Another method to fuse nformaton s to use a Kalman flter [Maybeck 79]. A Kalman flter can be used f the system that the flter wll model descrbes a lnear model and f the errors wthn ths model can be descrbed as whte Gaussan nose [Luo 89]. Whte nose means nose that s not correlated n tme, so nose predcton s mpossble [Maybeck 79]. The Kalman flter provdes an optmal estmated fltered value f you look at t from a statstcal pont of vew [Luo 89]. The Kalman flter s not very computatonally ntensve and can be appled n many dverse areas. The Kalman flter has been used many tmes wthn the RoboCup doman and often qute successfully [Detl 01, Roth 03]. 4.1 An archtecture for nformaton mergng To be able to merge observatons from robots n dfferent ways we had to thnk carefully when desgnng our mplementaton. Some of the requrements we had on the archtecture are presented below: A robot must be able to send observatons made by tself and receve observatons made by other robots. It must also be able to deal wth tme dfferences. The archtecture must support the ablty to merge observatons made by the robot tself wth observatons receved from other robots. It should be reasonably easy to add a new nformaton mergng method and change what flters to use. A robot should be able to flter observatons from tself even though t does not receve any observatons from the other robots and vce versa. In Fgure 4 we have extended the TCC archtecture wth the bascs of our mplementaton. To be able to handle and send vsual nformaton about objects n a general manner we created a structure called observaton wth nformaton about when an observaton was made, who made t, what object was observed, the dstance and angle to the object and the poston of the observng robot. We placed a module called InformatonSharng wthn the Wreless module that s responsble for sendng and recevng observatons to and from other robots usng the wreless LAN. We also created a smlar module wthn the Vson module called VsonFlterHandler responsble for creatng observatons made by the robot tself. To be able to handle these observatons and merge them together we needed a shared resource of some knd 11

16 and thus we created the FlterHandler that was placed n the shared memory resource called WorldState. Now both InformatonSharng and VsonFlterHandler can address ths handler. All observatons, both observatons made by the robot tself and receved observatons, are sent to the FlterHandler whch processes them. The FlterHandler holds a lst of FlterObjects, whch maps observed objects wth a specfc flter or nformaton merger. To clarfy ths even more we gve an example: When Robot1 sees the ball wth ts camera t creates an observaton of the ball n VsonFlterHandler and sends t to the FlterHandler. Next the FlterHandler fnds the FlterObject responsble for flterng observatons of the ball, whch n turn nserts the observed nformaton nto the specfc flter for the ball. Now f Robot2 has a vsual of the ball, t creates an observaton of the ball and sends t to Robot1 va the InformatonSharng module. When Robot1 receves ths nformaton n ts InformatonSharng module, t sends t to the FlterHandler for processng. Fnally the observaton made by Robot2 wll end up n the exact same flter as the prevously descrbed observaton made by Robot1 whch s necessary f we want to merge nformaton from many observers. Fgure 4. The fgure shows the extended TCC archtecture, where nformaton sharng and flterng are ncorporated. 12

17 4.2 Implemented flters We have created two dfferent types of flters or nformaton mergng methods, whch wll be tested and compared to fnd the most sutable one for the RoboCup doman. One type s of the more general knd and flters sngle varable data, for example n our case an x-poston on the feld used n RoboCup. These flters can be used to flter both global and egocentrc data but we have only used them for global data flterng. These flters are; "Exponental weghted movng average flter", "Lnear Regresson flter" and "Least mean square flter", shortened to Ewma, LRFlter and Lms respectvely. The other type s more specalzed and s specfcally used to estmate a poston of an object that s observed from more than one poston. These flters use a combnaton of global and egocentrc data when flterng. They are called; "Probablty matrx merger" and "Lne Intersecton merger", shortened to Pmm and Lm. We also have one helper flter used to stablze measurements of angles called "Angle group flter". From here on we wll refer to the flters by ther shortened name. The flters are here descrbed n further detal. Where t s needed the flters are descrbed wth a formula and below s an explanaton of what the symbols used n many flters mean. Flter specfc symbols are descrbed after each flter respectvely. = observaton number n = flter-length (number of observatons taken nto account when flterng) y = fltered value for observaton x = unfltered value for observaton w = weght for observaton t = tme when observaton was made Exponental weghted movng average flter In ths flter we have used the weghted average flter descrbed above wth some modfcatons. The major changes are that t has two separate weghts, one for how accurate each measurement s and one for how old the observed value s. The flter s called exponental because the tme weghts are exponental wth the newest values weghted hghest. These two weghts are then merged nto one sngle weght. The flter s called a movng flter snce t only flters the latest n number of values and thus constantly moves the flter to the newer measurements. We created ths flter because t was the most natural and easest nformaton mergng method. The major beneft s that t s very fast and relable. On the other hand, the smplcty of the flter mght be a dsadvantage when tryng to model a complex envronment such as RoboCup, where measurements are affected by qute a lot of factors. To calculate the tme weghts we used the followng formula: 1 tw = tme t To add the weghts together we used a smple multplcaton: w = aw * tw 13

18 Here s the man formula we used n ths flter: y x j j= n = j= n * w w j j tme = current tme aw = accuracy weght for observaton tw = tme weght for observaton Least mean square flter The Least mean square (Lms) flter comes from the sgnal processng area and s an adaptve flter [Haykn 91]. Lms s a flter consstng of two phases. In the frst phase the flter must be traned wth both some measured data and the actual desred data. The flter then calculates how much the measured data devates from the actual data and creates and stores weghts accordng to the errors and some addtonal constants. In the second phase, where the flter really s beng used, the only nput s the measured values and the flter uses ts estmated weghts to account for the errors of the measured values. One dsadvantage of ths flter s that you have to tran t wth some ntal data and that the data must represent the actual measured values whch wll be used n the second phase or the flter would not be able to account for the errors. Another dsadvantage of usng ths flter s that t s very dependent on a partcular constant that must be set properly or the Lms flter could gve very ncorrect results. If the constant s set too low, the flter would not be able to keep up wth changes of the measured values and f the constant s too hgh, the flter wll overdo ts compensaton of changes n the measured values and the result of the flter wll start to wobble uncontrollably. In the tranng phase the errors are frst calculated after whch the dfferent weghts based on the calculated error are calculated: e = d y = w 1 + * e 1 * x 1 w µ e = error (how far away from the actual value the fltered value s) d = desred (what the actual value s) µ = a constant n Lms, tells how much the flter should compensate for errors. Durng the actual flterng, the weghts from the tranng phase are used to calculate the fltered values: y = w * x j= n Lnear regresson flter j j We wanted to compare our other flters wth a predcton flter, whch s why we mplemented an nformaton mergng method that s based on the statstcal analyss method called lnear regresson [Körner 00]. It works lke ths: The method takes a group of values and tres to map the lnear functon: "y = a + b * x" to the values usng lnear regresson where the x s the tme and y s the measured value. When the other parameters of the functon have been estmated, the current tme s nserted nto the 14

19 functon and the estmated current value s calculated. The result of ths s the fltered value. In other words, the flter looks at the latest measured values and analyses them to see how they change accordng to tme and then predcts what the next value wll be. If the value beng fltered changes qute constantly accordng to tme ths flter should n theory perform well. Wthn the RoboCup doman such a constantly changng value could be the poston of the ball when t s beng kcked across the feld. One advantage of ths flter s that t as descrbed above predcts the next value dependng on the tme and thus should gve a good result even when the flter does not receve any measurements for a whle. On the other hand ths could also be a dsadvantage because the flter mght not keep up wth sudden changes n the measured values snce t predcts the next value. Another dsadvantage s that t s qute complex and could be computatonally ntensve. We wll here present the formulas for lnear regresson as we have used t. To make t easer to understand we have dvded the calculaton of constant b nto two formulas: Stt Stx = = Stx b = Stt ( t j ) j= n t j j= n x 2 j t j j= n n t j j= n * n 2 j= n To calculate the constant a, we used ths formula a = n x j j = n b n t j j = n x j We use ths lnear functon to calculate the fltered value usng the prevously calculated constants. y = a + b t Probablty matrx merger Ths flter was created to be able to estmate the most probable poston of an object from varous observaton postons wthout any knowledge of how far away an object currently s. Ths can be very useful f you for some reason cannot estmate the dstance to an object and thus cannot calculate a poston but only a drecton. It can also be useful when an estmaton of dstance s nosy and unrelable lke t can be n the RoboCup doman. The method creates a matrx representng the observed area, n our case the RoboCup feld, wth a probablty value n each cell. When the flter receves an observaton of an object, t draws a lne from the orgn of the observaton n the drecton of the observaton onto the matrx. Each cell wthn the matrx that the lne crosses gets an ncreased value of probablty. If two observatons lke these cross each other, there wll be a hgher probablty n the cell where the crossng took place. In our mplementaton we ntalzed the probablty matrx to zero probablty n each cell and ncreased t by a value of fve f there was an observaton. To estmate the poston of the object, the method evaluates all probablty values and returns the poston wth hghest probablty as the poston of the object. To deal wth old 15

20 observatons, the method terates through the entre matrx and decreases the value of probablty n each cell every now and then. In our mplementaton we decreased the probablty value wth 1 each frame. The obvous advantage wth ths flter s that t does not rely on some nosy dstance estmaton but on the other hand t wll probably requre a good trangulaton angle to work satsfyngly. Another dsadvantage of ths flter s that f you want hgh precson on the estmated poston you must have a large matrx, whch entals more data to evaluate and update each frame Lne ntersecton merger We dd not want to completely rely on one soluton to the problem of estmatng a poston from many observaton postons wthout any dstance estmaton, whch s why we decded to create another mergng method n case the frst one would fal to solve our problem for some reason. Ths method requres the same nput as the probablty matrx merger and gves the same type of result but t works dfferently. Ths method uses the observaton nformaton to create a representaton of a lne wth both a start and end poston. The flter terates through ts current set of observaton lnes and mathematcally calculates the ntersectons between all lnes. To get an estmated poston of the observed object the method takes the average of the postons represented by the ntersecton ponts. To deal wth old observatons, ths flter has a tmestamp lmt and throws away observaton lnes that are older than ths lmt. In our mplementaton we set ths lmt to 0.5 seconds. But even f an observaton from one of two robots s too old, the flter must keep t because t needs at least two observatons from dfferent locatons to be able to trangulate the poston of the observed object. Ths flter also has an outer boundary lmt whch means that the flter dscards postons of ntersecton ponts that are outsde ths lmt. We set ths lmt to the sze of the RoboCup feld; (-2200 mm, mm) to (2200 mm, 1450 mm) relatve to the centre of the feld. Ths s the formula the flter uses to merge the postons of the ntersecton ponts that are wthn the lmts. The formula s used for both x and y coordnates of each poston. n pk k = y = 0 n p k = poston coordnate calculated from two lne ntersectons for ntersecton k k = ntersecton number n = number of ntersectons Angle group flter When the dscovery was made that the angle estmaton to an observed object made by the robot was very naccurate and very often a total msreadng, we decded to create a flter for stablzng ths value. The flter was called AngleGroupFlter smply because t works wth groups of angles. All angles (0-360 degrees) are dvded n for example 20 groups wth 18 degrees n each group. If the robot estmates an angle to an object t s nserted n the group wth that angle. When a fltered angle s requested, the flter smply takes the average of all angles n the group wth the most number of angles n. Ths way, the angle can be stablzed wthout the rsk that a devatng value wll spol t. Ths flter was not actvated durng the experments, but s used n the unknown landmarks challenge. 16

21 4.3 Error sources There are a number of possble error sources that affect the nformaton that s ultmately shared. Measurements. Ths ncludes measurements of dstance and angle to dfferent objects on the feld. The dstance s manly measured by usng the camera of the Sony AIBO robot dog. By dentfyng known objects from examnng colours n the camera mages, the dstance s calculated by comparng the sze of the object n the mage wth the real world sze. However, lghtng condtons e.g. reflectons or the amount of lght can affect the nterpretaton of the mage, for example too much or too lttle lght wll make the dentfed object look lke t s further away or closer than t actually s. In Fgure 5 and 6 you can see two mages of a ball taken by a Sony AIBO ERS-7 robot dog n dfferent lghtng condtons. If the robot s calbrated to use brghter colours t mght calculate the dstance to the ball n the frst mage to a certan value. However, f the same calbrated colours are used n the second mage too, the robot wll probably estmate a longer dstance snce t s not as brght. In fact the robot mght not recognse the object at all snce the colours are so dfferent n ths partcular case. It s also possble that the object s partally occluded, whch makes the object look smaller and thus nterpreted as beng further away than t actually s. Errors n measurements are usually caused by random nose [Fukuda 96] and can be modelled as a probablty dstrbuton [Luo 89]. Ths probablty dstrbuton can be useful n nformaton mergng technques such as a Kalman flter. Localzaton. The robot localzes tself on the feld by measurng dstance and angle to landmarks around the feld. If these measurements are wrong, the poston of the robot wll not be accurate. Ths error s propagated to the locaton of every other object observed by the robot, whch f t s shared, dffers much from the other robots' opnons of object locatons. Identfcaton. Another error source s the dentfcaton of objects, and decdng whether two observatons from dfferent robots or sensors refer to the same object n the envronment [Luo 89]. The RoboCup doman normally contans known objects wth a known shape and colour, whch makes t easer to dentfy and tag observatons. However, dentfcaton errors can occur when the lghtng condtons are dfferent from the calbrated condtons. Ths sort of uncertanty can be represented as a confdence value of how confdent the robot s about the dentfcaton [Fukuda 96]. Fgure 5. Ball observed by a Sony AIBO ERS-7 robot under brght lghtng condtons. Fgure 6. Ball observed by a Sony AIBO ERS-7 robot under dark lghtng condtons. 17

22 5 EXPERIMENTS All tests that nvolved a robot were performed on a standard RoboCup football feld. For the tests that nvolved two robots and nformaton sharng, the robots were placed as shown n Fgure 7, facng each other. The robots constantly swept the feld n search of the ball or landmarks by turnng ther heads from sde to sde for a total of 180 degrees. When a robot saw the ball t started to follow t by turnng ts head n the drecton of the ball. The offcal ball of RoboCup was used. In the tests we wll refer to some flters wth the flter name and a number extenson, e.g. Ewma5. The number corresponds to the flter length,.e. how many pror observatons t stores. Pror to testng, we encountered a problem usng the Pmm flter that we dd not foresee. The problem was that f many observatons were made from roughly the same poston wth approxmately the same drecton, the cells close to the orgn of the observaton would be overrepresented snce many observatons wll go through these cells. As a result of ths, these cells would end up wth very hgh probablty values and the estmated poston of the object would be ncorrect n most stuatons. Based on ths, we regarded the flter as unusable and excluded t from all other tests except for the performance test Robots Landmarks Yellow goal Blue goal Fgure 7. General test setup. The two robots were placed on each sde n the mddle of the feld facng each other. 18

23 5.1 TCC Test In ths chapter, parts of the TCC archtecture were tested Localzaton test In ths test, the localzaton of the TCC archtecture was evaluated Purpose One of the error sources n nformaton sharng s the robot s estmaton of ts own poston. The poston estmaton of other objects n the world s drectly affected by the poston of the robot. If the estmaton of ts own poston vares greatly from the real world poston and t shares ts observatons of objects wth other robots, t wll share faulty data that wll be of doubtful use to the other robots. To test the TCC localzaton, we set up the followng test scenaro Procedure Localzaton test postons mm mm Localzaton test postons Corner landmarks Dagram 2. The dagram shows the test postons where the robot was placed to do localzaton. We used one Sony ERS-7 robot, whch we placed on measured postons on the football feld, whch you can see n Dagram 2. It localzed tself by turnng ts head from sde to sde n 180 whle searchng for and lookng at known landmarks on the feld, examnng ther sze and measurng the angles to the landmarks n an effort to estmate ts own poston. When we refer to the upper feld, we mean the fve postons on the rght, whle the lower feld are the fve postons on the left. The mddle feld refers to the three ponts n the mddle. 19

24 Analyss Localzaton test postons Corner landmarks (1500,850) (1500,0) (1500, -850) (750,0) (750,450) (750,-450) Vew ng drecton Dagram 3. Localzaton test - Upper feld. The robot was placed on the localzaton test postons, facng the yellow goal and rotatng ts head whle tryng to estmate ts own poston Localzaton test postons Corner landmarks FL (0,850) FR (0,850) FL (0, 0) FR (0,0) FL (0,-850) FR (0,-850) Vew ng drectons Dagram 4. Localzaton test - Mddle feld. The robot was placed on the localzaton test postons where t tred to estmate ts own poston. In the dagram, FR stands for facng rght, whch s n the drecton of the yellow goal, whle FL stands for facng left, whch s n the drecton of the blue goal. 20