Experiments in Outdoor Operation of RatSLAM

Transcription

1 Experments n Outoor Operaton of RatSLAM Dav Prasser, Goron Wyeth, Mchael Mlfor, School of Informaton Technology an Electrcal Engneerng Unversty of Queenslan St. Luca, Queenslan 4072 Australa {prasser, wyeth, mlfor}@tee.uq.eu.au Jonathan Roberts, Kane Usher CSIRO ICT Centre P.O. Box 883, Kenmore, Queenslan 4069 Australa {Jonathan.Roberts, Kane.Usher }@csro.au Abstract Ths paper shows ntal results n eployng the bologcally nspre Smultaneous Localsaton an Mappng system, RatSLAM, n an outoor envronment. RatSLAM has been wely teste n noor envronments on the task of proucng topologcally coherent maps base on a fuson of oometrc an vsual nformaton. Ths paper etals the changes requre to eploy RatSLAM on a small tractor equppe wth oometry an an omnrectonal camera. The prncpal changes relate to the vson system, wth others requre for RatSLAM to use omnrectonal vsual ata. The ntal results from mappng aroun a 500 m loop are promsng, wth many mprovements stll to be mae. 1 Introucton RatSLAM s a bologcally nspre system for smultaneous localsaton an mappng (SLAM) 1. RatSLAM has been eploye successfully on Poneer 2 moble robots [Mlfor an Wyeth, 2003; Mlfor et al., 2004; Prasser et al., 2004] usng a camera as an external sensor for mappng the envronment. To ate all work wth RatSLAM has been conucte wth forwar facng cameras wth narrow fels of vew (< 50 ). In ths paper the ntal steps towars mgratng RatSLAM to an outoor robot equppe wth an omnrectonal camera are scusse. The majorty of ths work s occupe wth changes to the vson system an the vsual learnng process n orer to beneft from the ncrease fel of vew. 1.1 RatSLAM RatSLAM s base on technques use n computatonal 1 Ths research s sponsore n part by an Australan Research Councl grant. moels of the roent hppocampus. The system contans three man moules: Pose Cells (PC); Local Vew (LV); an Path Integraton (PI). The Pose Cell moule s the heart of the system; t s an array of neural unts that mantans the robot s pose estmate. Each unt n the PC can be consere to correspon to a regon n (x, y, ) space. The more actvate a unt becomes, the more RatSLAM beleves t s near that unt s poston. The Path Integraton moule s responsble for ajustng the actvate Pose Cells base on the robot s sense of self moton. Fnally, the Local Vew represents the external sensors of the robot, n the form of another array of neural unts. RatSLAM uses Hebban learnng to assocate actve LV unts wth actve PC unts to bul a map n the form a set of vson pose corresponences, whch can later be use to ajust PC actvty. The LV unts respon to some form of cue n the robot s camera mages. In the past, ths has been the presence of partcularly coloure objects or sgnfcant eges n the envronment. 1.2 Appearance Base Vsual Learnng RatSLAM assocates the vsual scene wth a poston. RatSLAM oes not nee a complcate analyss of the envronment, prove that the Local Vew representaton s the same or smlar each tme the robot vsts a partcular locaton. Ths can be accomplshe by makng LV unts respon to partcular vsual appearances. The most recent RatSLAM vson system memorse vews as t travelle through the envronment [Prasser et al., 2004]. Each LV unt correspone to one of the learnt vews an was actvate when the camera mage matche the unt s learnt vew. Ths system worke qute well an the new outoor system shoul beneft from functonng n the same way. The noton of learnng the robot s vew an assocatng t wth place has been use before n bologcally nspre robots [Arleo et al., 2001], an n other robot localsaton systems, n partcular systems usng panoramc or omnrectonal cameras [Gonzalez-Barbosa an Lacrox, Ulrch an Nourbakhsh]. These systems for learnng 1

2 panoramc vews have use hstogram matchng. In a hstogram matchng scheme mages are represente by a hstogram of some set of attrbutes, for example hstograms of greyscale brghtness, colour, or ege recton an ntensty. The prmary avantage of hstogram matchng s that when the hstogram s compute the nformaton about the poston of features n the mage s scare. Snce rotaton of an omnrectonal camera about ts optcal axs results n a rotaton of the mage about ts centre, the hstogram escrpton of the mage shoul not change as the camera rotates. Gven that, on moble robots, the camera s man axs s parallel to the robot s rotatonal axs, then a hstogram escrpton of an mage shoul be nvarant to the rotaton of the robot. Ase from the nvarance to robot rotaton, hstograms have the other useful propertes: compact representaton of mages; a small level of translaton nvarance; an a number of convenent matchng metrcs, 2 nclung matchng [Gonzalez-Barbosa an Lacrox, Ulrch an Nourbakhsh]. These propertes make the hstogram matchng of vsual appearance an attractve opton for the eployment of RatSLAM on a robot wth an omnrectonal camera. 1.3 Summary of Paper Ths paper s prmarly concerne wth the steps neee to make RatSLAM work wth outoor ata. The most sgnfcant changes were mae to the vson system whch s outlne n the next secton. Other changes to RatSLAM are etale n secton 3 whle the outoor robot an other aspects of the expermental setup are escrbe n secton 4. RatSLAM trajectores an other results are n secton 5 an conclusons an a summary of further work are n secton 6. 2 Vson System The new platform, an autonomous tractor, s equppe wth an omnrectonal camera, provng approxmately 320 of vsual coverage aroun the robot wth a small bln spot facng towars the rear of the robot (Fgure 1). Fgure 1: Robot s vew of ts envronment through an omnrectonal camera, the sensor that RatSLAM uses to mantan ts localsaton. The robot s facng towars the top of the mage. The support for the mrror obscures part of the robot s fel of the vew towars ts rear. The large level of vsual coverage proves an opportunty to generalse from one robot pose to another. When RatSLAM was use wth 50 fel of vew cameras, t was unable to extrapolate from one vew to another. Ths force the robot to travel to the new poston to etermne ts vsual appearance. Ths s partcularly troublesome for robot rotaton, for example whle traversng a corror n one recton RatSLAM s unable to gather the nformaton neee to re-localse when travellng n the opposte recton (except by usng a turn-back-an-look behavour). 2.1 System overvew The vson system has four basc steps. Frstly, the mage s converte to a panoramc representaton an normalse n ntensty (Fgure 2). A hstogram of the panoramc mage s ntensty s constructe an use to represent the vsual scene to a matchng mechansm. Ths hstogram s compare to a learnt lbrary of reference hstograms to prouce a shortlst of canate vews. Hstograms that are suffcently stnct from those alreay learnt are ae to the lbrary, as s the panoramc mage that prouce the hstogram. Fnally usng the learnt panoramc mages the orentaton of the robot relatve to when each of the canate vews was learnt can be recovere. Fgure 2: A panoramc mage constructe from Fgure 1. The area near the centre of Fgure 1 has been scare as t contans lttle nformaton useful for re-localsaton. The area of the panoramc mage contanng the camera bln spot or ea zone has been also croppe. The mage shows a vew of about 320 aroun the robot. Hstogram Matchng By usng a rotaton nvarant representaton of the camera nput, t s possble to bul a vsual scene matchng system smlar to that alreay use successfully wth RatSLAM. A well emonstrate vew matchng metho for omnrectonal cameras s Hstogram Matchng. Whle many fferent mage attrbutes can be nclue n the hstogram matchng scheme, for the tme beng RatSLAM uses only normalse greyscale, whch s matche usng the Jeffrey vergence. The 2 statstc was orgnally use but t behaves baly when a hstogram bn has a zero entres. Jeffrey vergence has prevously been foun to be a goo metrc for hstogram base localsaton [Ulrch an Nourbakhsh, 2000]. For two hstograms, H an K, wth entres h an k, the vergence s: j 2h 2k, (1) ( H K ) = h log + k log h + k h + k The vergence s calculate between the hstogram of the current camera mage an all of the learnt hstograms. Learnt hstograms whose vergence s below a threshol are consere to be reasonable matches an are sent as canates to the next stage of the vson system. When there are no matches aganst the learnt hstograms then the current hstogram s memorse. In ths way the robot wll traverse ts envronment learnng places that are vsually stnct from each other. Each hstogram effectvely 2

3 represents a small area of the envronment. However snce hstogram matchng uses vsual appearance t s possble that one hstogram may match many separate locatons n the envronment. Recovery of Orentaton If only the hstograms are store then by usng a rotaton nvarant recognton scheme we have lost the ablty to recover the robots orentaton urng re-localsaton. The soluton s to store the mage that generate the hstogram. After recognsng a scene by usng hstogram matchng the reference mage can be matche aganst the current camera mage for varous hypothetcal rotatons of the robot. The best ft wll ncate the fference n robot orentaton from when the reference mage was acqure. Determnng rotaton can be best accomplshe from the panoramc mage. In the unwarpe mage each column of pxels correspons to a partcular bearng, so smulatng a change n orentaton of the robot can be accomplshe smply by shftng the mage n the approprate recton. Atonally the panoramc mage oes not retan the centre part of the omnrectonal mage. Ths area s not thought to be useful for localsaton as t usually contans only the roa near the robot. In the current system the panoramc mages are reuce n resoluton to lower the computatonal loa of the matchng process. The learnt mage n memory s rotate through 360 n 36 screte steps an compare to the current camera panoramc mage usng a Sum of Absolute Dfferences (SAD) metrc. The number of rotatonal steps taken to fn the best ft s reporte by the vson system to the rest of RatSLAM, whch uses t as a coarse measure of bearng. A porton of the full panoramc mage s a ea zone create by the support for the mrror. Ths area rotates wth the robot an appears statc n all camera mages so t cannot be use n the matchng process. The poston of the ea zone must be tracke when rotatng mages to ensure that the matchng metrc s not compute for ths area n ether the camera mage or the learnt template mage (Fgure 3). Ths wll ntrouce a bas towars makng matches where the two ea zones are not overlappng, snce n these crcumstances the SAD s calculate over a smaller area. Normalsaton by the number of pxels that contrbute to the matchng solves ths problem. Camera Image Fgure 3: When rotatng the template mage to etermne at what orentaton t best matches the current camera mage the camera s ea zone must be taken nto account. The robot has no knowlege of what les n the ea zone n ether the current or template mages so these areas cannot be compare. D Z Template Image D Z 0 Template Image 3 Alteratons to RatSLAM Apart from the new vson system there were some other changes that neee to be mae to RatSLAM. The most sgnfcant of these were mae to the vsual assocaton process n orer to agan to take avantage of the omnrectonal camera. The spatal resoluton that RatSLAM operates at was also reuce. 3.1 Local Vew In the RatSLAM archtecture the Local Vew (LV) cells represent the robot s external sensors to the rest of the network. As n recent noor RatSLAM work each LV cell correspons to one learnt template. The actvaton of the cell, V, ncreases as the fference between the camera mage an the template,, ecreases: 1 V = ( + ε ) 0 > Where max s a senstvty parameter an prevents numercal blow out. Fnally the actvaton of all of the LV cells are normalse to unty. The orentaton of the robot relatve to the template also nees to be expresse to the rest of RatSLAM; ths s accomplshe by a set of offset nces that accompany the LV cells. These offsets are the number of steps that the template mage was rotate by to acheve the best match. 3.2 Vsual Assocaton RatSLAM assocates Local Vew unts wth actvate unts n the three mensonal Pose Cell array (Fgure 4). The orentaton of the mage s also store urng the vsual assocaton process by etermnng whch pose cell woul be actvate after unong the rotaton ncate by the vson system. In other wors, the actvate LV unts are assocate wth the pose cells that shoul be actve f the mage was etecte at no offset. If an actve pose cell j locate at orentaton j s to be assocate wth LV unt, an offset,, then the LV unt s actually assocate wth a pose cell unt locate at the same poston but at orentaton j : ' j θ = θ γ j Smlarly, when re-localsng, the LV njects energy to the pose cells that have been assocate wth the LV but shfte n the recton by an amount controlle by the LV unt s offset: j j max max (2) (3) θ = θ ' + γ (4) The vson system reports offsets n the nteger range of zero to 35, whch s the same as the number of cells n the RatSLAM system s axs. Ths rastcally smplfes the process of computng the array nex of the pose cell after rotaton. 3

4 V () β ()(lmn) Y P (lmn) Fgure 4: The local vew network an pose cell network. Actvate unts n the local vew, V, become assocate wth actvate unts n the pose cells, P, through learnt weghte connectons between the two networks. The relatve orentaton of the robot reporte by the vson system s use to ynamcally change the Pose Cells each LV-PC lnk s connecte to. Theta X 4.1 Data Sets Two ata sets were acqure from the robot as t traverse a large out oor loope roa approxmately half a klometre n length. In the frst experment the robot complete two loops n a clockwse recton, whle n the secon the robot complete one traversal n each recton. There was a tme elay of about 45 mnutes between the frst an secon atasets. Camera mages were acqure at 1 Hz an robot oometry at about 15 Hz. The robot s path takes t through both bult up areas an unstructure natural areas. Ase from changes n lghtng cause by clous obscurng the sun, there are also humans movng aroun from tme to tme n the robot s fel of vew. These changes make the vsual envronment ynamc rather than statc. 4 Expermental Setup The experments escrbe n ths paper were performe offlne on ata acqure by the robot urng several traversals through an outoor envronment. The offlne processng of ata was a matter of expermental convenence; the vson an RatSLAM algorthms can be mae to run n real-tme. The outoor robot s a tractor evelope by the CSIRO [Usher et al, 2003]. Ase from the omnrectonal camera an oometry sensors the tractor s also equppe wth a laser range fner an compass, nether of whch are use by RatSLAM. Start Fgure 6: Approxmate trajectory of the robot urng ata acquston. The arrows ncate the path of the robot urng the frst lap of the envronment. 4.2 System Parameters The panoramc mages are constructe wth a resoluton of 0.01 ra/pxel an occupy a sol angle of 5.6 ra horzontally by.52 ra vertcally. The hstograms have 64 bns an a Jeffrey vergence of 300 s consere to be the maxmum vergence before learnng a new hstogram. The mage matchng s conucte at a lower horzontal resoluton, where each pxel correspons to 10 horzontally. If there was no camera ea zone then each reference mage woul pxels n sze. Ths resoluton s chosen to smplfy the mage rotaton process. The Sum of Absolute Dfferences matcher uses a fference of less than 150 as a match. Most vson parameters were selecte by estmate an the only parameter that was extensvely tune was the hstogram vergence. The RatSLAM vsual assocaton process ha to be mae weaker to reflect the ncrease ambguty n the vson system. Fnally RatSLAM pose cell resoluton was ncrease from 250 mm to 2000 mm, reflectng the change n scale from prevous noor envronments to the new larger settng. Fgure 5: The autonomous tractor. The omnrectonal camera an mrror assembly s mounte on an elevate pont above the front wheels. 5 Results Two man results are acqure from the experments: the Local Vew actvty an the Pose Cell trajectory. RatSLAM mantans an approxmately Cartesan mappng of the envronment wth an emphass on consstent an repeatable results rather than physcal accuracy. A goo result for RatSLAM s when repeate robot trajectores n the real worl correspon to repeate paths of actve Pose Cells. 4

5 5.1 Vson Results The actvate Local Vew unts urng the experment are shown n Fgure 7. The system begns wth no learnt LV unts an numbers them n the orer that they are learnt, causng the frst rsng lne n Fgure 7. At 300 secons nto the experment the robot has complete one crcut of the loop an begns a secon one, at ths pont prevously learnt hstograms are foun agan an the LV unts begn to be actvate agan n the same pattern as on the frst lap. The system contnues to learn new hstograms an recrut more LV unts on the secon lap, although at a reuce rate. Ths s partly because the some parts of the envronment have change vsually between laps but also because the system has been set to a low level of generalsaton. The contnual ncrease n the number of learnt hstograms wll eventually fnsh once all of the lkely vews of the envronment are learnt. LV Cell Number Tme [s] Fgure 7: Actvate Local Vew (LV) unts versus tme as the robot traverses the envronment. For any gven nput zero or more templates may be foun. Begnnng at 300 secons the robot begns to retrace ts path an re-encounter prevously seen vews, resultng n repeate patterns of Local Vew actvty. 5.2 Trajectory Results The RatSLAM trajectory results for the forwar path are shown n Fgure 8. Ths fgure shows the trajectory of both laps supermpose upon each other. There s a very goo overlap of the secon lap s trajectory upon the frst ncatng the network s usng vsual nformaton to contnuously correct ts beleve poston n the worl. RatSLAM s also able to recognse the loope nature of the path an ajust ts poston to account for the oometry errors that accumulate. Ths can be seen when after completng the frst lap RatSLAM changes ts beleve poston from pont a to pont b. Fgure 8: RatSLAM trajectory whle completng two forwar traversals of the route shown n Fgure 6. Durng the secon lap the RatSLAM recognzes that t s repeatng a learnt path an ajusts ts perceve poston from a to b. Re-localsatons are shown as thn lnes. The system oes make one error early n the frst lap where t ncorrectly re-localses from pont c to pont b. Fgure 9 shows camera mages taken at both of these ponts an t s easy to see why these two ponts coul be msclassfe, especally when the mages are compare n such a coarse way. Usually RatSLAM gnores short term vson errors lke ths, however ths error persste long enough for RatSLAM to ece that there was suffcent evence to relocalse. Ths s more an error on the part of the vson system than of the SLAM component. In orer to tell the mages n Fgure 9 apart a more etale escrpton wll be neee than just a hstogram of grey scale ntensty. Fgure 9: Panoramc mages acqure at pont b (top) an pont c (bottom) n Fgure 8. A consstent msclassfcaton of the bottom mage causes RatSLAM to re-localse from c to b. 5.3 Travel n Both Drectons As a fnal test both ata sets were combne nto one large run n whch the robot travelle n both rectons along the loop. Ths s a partcularly ffcult ata set as the en of the frst run oes not conce wth the start of the secon run effectvely causng a global knap of the robot. At the same tme a large tme wnow has elapse unbeknownst to the robot urng whch the vsual envronment may have change. The trajectory of the combne run s shown n Fgure 10. The man purpose of ths experment s to test the ablty of RatSLAM to localse the robot whle travellng n a novel recton along a route that has alreay been learnt. c b Start a 5

6 c Fgure 10: RatSLAM trajectory whle completng three forwar an one backwars traversals of the route shown n Fgure 6. The robot performs a 180 turn at the pont a an mantans a consstent path up to pont, espte travellng n the opposte recton to the one t has experence. On the thr an fourth crcuts the robot s unable to mantan ts localsaton all of the tme resultng n the ncorrect path sectons b-c an -e. The robot mantans goo localsaton on the reverse path from pont a to pont, although t loses track of ts poston after pont untl pont e. Ths emonstrates that by usng an omnrectonal vson sensor RatSLAM can generalse about ts orentaton. Prevously RatSLAM Pose Cell trajectores woul not necessarly have overlyng forwar an reverse paths. On the thr an fourth laps RatSLAM fals to reman localse between b an c an between an e. Ths regon of falure appears to have ha a large varaton n lghtng between the frst two an the last two laps. The area can be roughly characterse beng a natural envronment whle the areas where localsaton was well mantane coul be escrbe as bult up areas. If the experment ha been performe on a contnuous ata set an over a shorter nterval of tme the results woul have been better. The result hghlghts the fact that the current brghtness normalsaton process s naequate for ealng wth long term lghtng varatons. Correctng ths woul be necessary to gve the system the capablty to operate outoors for extene peros of tme. 6 Concluson We have shown that t s at least feasble to transfer the exstng RatSLAM system to an outoor envronment. The major changes were to the vson system to take avantage of the omnrectonal camera. The vson system stll operates uner an appearance base scene learnng scheme albet usng a fferent set of prmtves to prevous RatSLAM vson systems. There are ponts of falure n the system that coul be rone out by aoptng a rcher set of feature etectors. As expecte hstogram matchng proves a quck metho of vsual appearance learnng that s useful for robot localsaton an mappng. b e a 6.1 Future Work The system coul be expane to use other mage attrbutes such as colour n the hstogram recognton stage. Other hstogram matchng technques such as Earth Mover s Dstance shoul be nvestgate as well as more sophstcate methos for ensurng llumnaton nvarance. Also the system can be easly blene wth the ege etecton scheme prevously use wth RatSLAM [Prasser et al., 2004] at both the hstogram stage an the orentaton recognton stage. Ths wll be the mmeate next stage of evelopment. RatSLAM s capabltes coul also be teste on less topologcally structure outoor envronments, for example, large open areas wthout obvous paths or routes. Fnally, RatSLAM coul be fully eploye to operate on the robot n real tme. References [Arleo et al., 2001] Angelo Arleo, Fabrzo Smeral, Stéphane Hug, an Wulfram Gerstner, Place Cells an Spatal Navgaton base on Vson, Path Integraton, an Renforcement Learnng. Avances n Neural Informaton Processng Systems, [Gonzalez-Barbosa an Lacrox, 2002] J.-J. Gonzalez- Barbosa an S. Lacrox. Rover localzaton n natural envronments by nexng panoramc mages. Proceengs of the Internatonal Conference on Robotcs an Automaton 2002, IEEE, [Mlfor an Wyeth, 2003] Mchael Mlfor an Goron Wyeth. Hppocampal Moels for Smultaneous Localsaton an Mappng on an Autonomous Robot. Proceengs of the Australasan Conference on Robotcs an Automaton (ACRA), [Mlfor et al., 2004] Mchael Mlfor, Goron Wyeth, Dav Prasser RatSLAM: A Hppocampal Moel for Smultaneous Localzaton an Mappng. Proceengs of the Internatonal Conference on Robotcs an Automaton, n press, IEEE, [Prasser et al., 2004] Dav Prasser, Goron Wyeth an Mchael Mlfor. Bologcally Inspre Vsual Lanmark Processng for Smultaneous Localzaton an Mappng. Proceengs of the IEEE/RSJ Internatonal Conference on Intellgent Robots an Systems (accepte), [Ulrch an Nourbakhsh, 2000] I. Ulrch an I. Nourbakhsh. Appearance-base place recognton for topologcal localzaton. Proceengs of the Internatonal Conference on Robotcs an Automaton 2000, IEEE, [Usher et al, 2003] Kane Usher, Peter Corke an Peter Rley, Vsual servong of a car-lke vehcle -- an applcaton of omnrectonal vson, Proceengs of the Internatonal Conference on Robotcs an Automaton 2003, IEEE,