Increasing multi-trackers robustness with a segmentation algorithm

Transcription

1 Increasing muli-rackers robusness wih a segmenaion algorihm MARTA MARRÓN, MIGUEL ÁNGEL SOTELO, JUAN CARLOS GARCÍA Elecronics Deparmen Universiy of Alcala Campus Universiario , Alcalá de Henares. Madrid. SPAIN hp:// Absrac: - Differen soluions have been proposed for muliple objecs racking based on probabilisic algorihms. In his paper, he auhors propose he use of an only paricle filer o rack a variable number of objecs. The robusness and adapabiliy of he esimaor are increased by he use of a clusering algorihm. The measuremens used in he racking process are exraced from a sereovision sysem, and hus, he 3D posiion of he racked objecs is obained a each ime sep. Tracking resuls are presened a he end of he paper, showing he reliabiliy of he proposals. Key-Words: - Probabilisic, Muli-racking, Robusness, Clusering, Paricle Filers. 1 Inroducion Probabilisic algorihms in heir differen implemenaions (Muli-Hypohesis Techniques - MHT- [1], Paricle Filers -PF- [2] and heir diversificaions [3][4]) have fully shown heir reliabiliy in esimaion asks. Nowadays hese mehods are widely applied o solve posiioning problems in robo auonomous navigaion [5]. The idea of racking muliple objecs appeared wih he firs auonomous navigaion robo, o overcome he obsacle avoidance problem and soon probabilisic algorihms, such as PF ([6], [7]) and KF ([8]), were applied o achieve his aim. The objecive is, in any case, o calculae he poseriori probabiliy r r ( p( x y 1 : ) ) of he sae vecor in he recursive wo seps sandard esimaion process (see (1)), in which, a leas, some of he involved variables are sochasic, and by means of he Bayes rule: r r r r r r r r r p( x y: ) = p( y x ) p( x x 1) p( x 1 y1: 1) x 1 η (1) To solve he mulipliciy problem, an expansion of he sae vecor o include ha of all he elemens o rack (see (2)) was he firs soluion proposed in [9]. r r r r k 1 2 k i χ = x, x,..., x = x (2) { } U r i= 1 The compuaional load of he resulan esimaor does no allow achieving a real ime execuion of he algorihm for more ha 4 or 5 objecs [10]. Anoher soluion for he muliple objecs racker is o use a sandard esimaor o rack each objec bu, apar of he inefficiency of he final algorihm, i canno deal easily wih a dynamic number of objecs. On he oher hand, in order o work wih independen esimaors i is necessary o include an associaion algorihm o assign each measuremen o one of he rackers. Mos of he associaion soluions are based on he Probabilisic Daa Associaion (PDA) heory [11], such as he Join Probabilisic Paricle Filer (JPDAF) like in [12]. Again, he problem relaed o hese echniques is he execuion ime. In his conex he auhors propose in [13] anoher soluion o he muli-racking problem based on a single PF whose mulimodaliy is exploied o perform he muli-racking ask. The algorihm is called Exended Paricle Filer (XPF). This soluion has been esed in complex indoor environmens wih sonar [13] and sereo-vision daa [14] wih good resuls. A clusering algorihm is used in his poin o organize he measuremens and he informaion involved in he esimaion process, and o increase he robusness and reliabiliy of he final esimaor. In his paper, a general revision of he racking sysem developed, based on sereovision measuremens, is presened, a sligh descripion of he XPF developed in [13] is hen included; he funcionaliy of he wo proposed clusering mehods is presened; and finally some resuls and conclusions obained from he segmenaion processes applied o he probabilisic esimaor are exposed. 2 General descripion Mos of he racking sysems developed in he las years for auonomous navigaion and surveillance applicaions are based on visual informaion. Measuremens obained from a vision sysem are generaed boh by he environmen and he objecs o rack. To use his informaion in he racking ask a process o disinguish he daa se ha comes from he

2 objecs o rack is needed. This pre-processing algorihm is especially imporan o achieve a real ime execuion, due o he big amoun on measuremens included in he vision image. Fig. 1 shows he funcional block diagram of he global racking sysem in which a classificaion preprocess has been included o organize he measuremens ha come from he vision sysem in wo groups: hose ha inform abou he posiion of he objecs in he image. hose ha are generaed by he elemens in he environmenal srucure (such as floor and walls). Wih he menioned funcionaliy, he classificaion pre-process behaves as an obsacle deecion module. As i has been exposed in Fig. 1, measuremens classified as environmenal srucure ones can be used o implemen a parial reconsrucion of he environmen in which he racked objecs and he robo iself are moving. This process, ha is independen of he aking applicaion, is sill under developmen by he auhors. 3 The classificaion algorihm Developing obsacle avoidance asks requires 3D informaion abou he objecs posiion in he environmen. To obain hese daa, a sereo-vision sysem has been used as shown in Fig. 1. Fig. 2 shows he process used o deec and exrac he array of measuremens from he sereo-vision sysem, which is explained in he following ex. The sereo-vision sysem proposed is formed by wo black and whie digial cameras whose acquisiion is synchronized. The 3D poins are obained from each pair of frames, by means of an epipolar echnique. As he amoun of informaion in each image is oo big o be compued i in real ime, maching pixels for he epipolar equaion are aken among he edge image obained applying a canny filer o he inpu frame. Sereo-vision Sysem Classificaion: Objec/Srucure Obsacle Deecion Probabilisic Muli-racker Environmen Parial Reconsrucion Figure 1. Block diagram of he global racking sysem Lef + Righ Seq. Frame = Lef + Righ Canny in Lef ROI Hough in Lef Canny image Erasing poins from long lines Epipolar Maching (ZNCC Correlaion) XZ Projecion Neighbourhood Filer 2 Files: Objecs/Srucure Figure 2. Block diagram of he sereo-vision classifier and objec deecor Wih he pixels in he edge image, he classificaion process is performed. Differen sraegies have been esed o achieve he bes resuls classifying he measuremen se, boh in execuion ime and in reliabiliy. Edges corresponding wih environmenal srucures have he common characerisics of forming long lines, and hus, he Hough ransform has been seleced as he bes procedure o erase he pixels from he edge image ha should no be par of he objec measuremen array. The execuion ime of he developed classifier is around 60ms, which is an accepable soluion o implemen a real ime acquisiion image process of 15fps o 33fps. Differen soluions are sill under developmen o achieve a more robus classificaion process. 4 The muli-racking algorihm As menioned in he inroducion, a PF is used as a mulimodal racker o esimae he posiion and speed of he objecs in he environmen, from he measuremen array obained in he classificaion process. PF is a paricularizaion of he Bayesian esimaor in which he densiies relaed o he poserior esimaion (also called belief) is discreized. A deailed descripion of he PF funcionaliy and mahemaical base can be found in [2] and [4]. As he sae vecor is no discreized, like i is in mos of he res Bayes filer implemenaions, he PF is more exac in is esimaion han he KF or esimaors based on a grid (MoneCarlo esimaors). Moreover, due o he same reason, he compuaional load of his Bayes filer form is smaller han in oher implemenaions, and hus, more adequae o implemen real ime esimaion. x2 x2

3 Finally, PFs include an ineresing characerisic for muli-racking applicaions: he abiliy of represening muliple esimaion hypoheses wih an only algorihm, hrough he mulimodaliy of he belief. This faciliy is no available in he opimal implemenaion of he Bayes esimaor, he KF. For all hese reasons, he PF has been hough as he mos appropriaed algorihm o develop a muli-racking sysem. 4.1 The XPF Mos of he soluions based on a PF presened in he relaed scienific lieraure do no use he mulimodal characer of PF o implemen he muli-racker. These alernaive soluions, menioned in he inroducion of his paper, are however less efficien in execuion ime. In order o adap he sandard PF for is use racking a variable number of elemens, some modificaions mus be included in he basic algorihm. In [13] and [14], he auhors propose wha is called he exended paricle filer (XPF), an adapaion of he sandard PF for racking a variable number of elemens. The XPF firs approach appears in [15]. The algorihm proposed here was neverheless no used in real racking asks because i is no robus enough. The XPF proposed by he auhors in [14] includes a clusering algorihm o improve he behaviour of he firs exended PF, giving as a resul he XPF-CP process, whose funcionaliy is shown in Fig. 3. The clusering algorihm whose funcionaliy is presened nex in his documen, organize he daa in clusers ha represen all objecs in he scene. These clusers are hen used in he racking algorihm. Wih he clusered daa wo imporan innovaions are included in he sandard PF in order o faciliae he muli-racking process: a) Re-iniializaion sep: M from he N oal number of paricles ha conform he belief disribuion in he PF are direcly included in his sep previous o he predicion sep. Wih his sep, measuremens relaed o newly appearing objecs in he scene have a represenaion in he esimaor belief. To improve he robusness of he esimaor, he insered daa are no seleced randomly from he array bu from is segmenaion. Choosing measuremens from every cluser ensures a probable represenaion of all objecs in he scene, and herefore he robusness of he muli-racker is increased. Thanks o his re-iniializaion sep he poserior disribuion dynamically adaps iself o he differen objecs presen in he scene. b) Resampling sep: This sep is also modified from he one in he sandard PF. On one hand, only (N-M) samples of he belief have o be exraced in his sep, as he M resing paricles would be insered in he re-iniializaion sep. On he oher hand, he clusering process is also used in his sep, because he imporance sampling funcion used o weigh he paricles is obained from he similariy beween each one of hem and he cenroid of he clusered inpu daa se. Using he cluser cenroids o weigh he paricles relaed o he newly appeared objecs is increased, improving he robusness of he final esimaor. Wihou he clusering process, he soluion proposed in [15] rejecs he hypohesis relaed o new objecs, and hus, he mulimodaliy of he PF canno be exploied. The robusness problem has been he main reason in he researching communiy for no using he PF mulimodal capabiliy. Wih he algorihm proposed in his paper, a new soluion for muliple racking using an only esimaor is designed. S ' S W N-M paricles from poserior (black) M/k new paricles randomly aken from each hisory buffer (whie) Paricles propagaion (cluser propagaion is also done o improve he clusering efficiency) Imporance Sampling The mos similar cluser is also used o obain each paricle likelihood Y Clusering new measuremens (whie) from updaed clusers a -1 Hisory buffers (black) are reconsruced wih new measuremens + similar hisoric members '' S Resampling N-M paricles according o heir cluser associaion Figure 3. Descripion of he XPFCP funcionaliy

4 4.2 Clusering Two differen algorihms have been developed for clusering he se of measuremens. Is reliabiliy is similar, hough he proposal is o selec which o use depending on he environmenal siuaion. Fig. 4 and 5 show heir funcional descripion. A more deailed analysis of hese algorihms can be found in [16]. The clusering process presened in Fig. 4 is based on he deerminisic k-means algorihm, which has been modified o handle a variable and iniially unknown number of clusers. A clusering updaing pre-process is included in he algorihm in order o minimize is execuion ime. Fig. 5 describes he second proposed clusering algorihm, which is a fuzzy/probabilisic classifier based on he sandard subracive one. This second opion has a longer execuion ime, hough i has he advanage of giving secondary informaion abou he clusers found in he inpu daa, such as is likelihood. This exra informaion is very useful in he PF based esimaion process. A validaion process is added o he clusering algorihm in order o increase he robusness of he global algorihm o spurious measuremens. The validaion process funcionaliy (shown in Fig. 6) is he following: When a new cluser is creaed, i is convered ino a candidae ha will no be used in he probabilisic algorihm unil i is possible o follow is dynamic evoluion. The same procedure is used o erase a cluser when i is no confirmed wih new measuremens for a specific number of imes. Ini: Cluser Updaing Calculae he dis from each member o all cenroids dis>dismax Assign he member o he mos similar cluser Recalculae cluser cenroides For all members While cluser changes Invalidae clusers ha have los all heir members Creae new cluser Ini: Densiy of each member Selecion of he bigges densiy: pdfmax pdfmax>limi Creae new cluser pdf(cenroid)=pdfmax Recalculae all densiies End Figure 5. Descripion of he subracive clusering algorihm CluserValid++ CluserValid == IsValid CluserValid=IsValid Calculae validaion parameers Valid? End CluserValid-- CluserValid == IsInValid Erase he cluser Figure 6. Descripion of he validaion process for he clusering algorihm 5 Resuls and Conclusion The global racking algorihm based on a sereovision sysem shown in Fig. 1 has been implemened in a mobile 4-wheeled plaform. Differen ess have been done in unsrucured indoor environmens, whose resuls are shown in his secion. The sereo-vision sysem is formed by wo black and whie digial cameras synchronized wih a Firewire connecion and locaed in a saic mouning arrangemen, wih a gap of 30cm beween hem, and a a heigh of 1.5m. The classificaion and racking algorihms are run in a PIV wih 512Mbyes of RAM, wih a global execuion ime around 80ms. Wih hese daa, a real ime muli-racking sysem is developed a a speed of more han 10fps. Fig. 7 shows he oupu of he classifier in one of he experimens, wih 5 frames of a global sequence. Classificaion resuls for each one are displayed in 3 images organized verically: End Figure 4. Descripion of he k-means clusering algorihm

5 Figure 7. Resuls of he classificaion algorihm in a real siuaion The one on he op shows he edge image obained direcly from he lef frame. Boh obsacles and environmenal srucure borders are mixed in he image. The picure in he middle shows he resul of he classificaion process in he XY projecion (x range is from -5m o 5m, and y range is from 0.2m o 20m). Whie dos represen he measuremens ha, according o he classifier resuls, belong o obsacles. The boom one shows he final lef image in which poins classified in he obsacle measuremen are highlighed wih a colour according o he heigh where hey are measured. From his figure, i can be concluded ha he classificaion objecive has been achieved. The resuling obsacle daa se can now be insered in he probabilisic esimaor. I is ineresing o poin ou ha he number of objecs presen in each frame canno be easily exraced direcly from he measuremen daa se. Therefore, he probabilisic racker will be necessary o deermine i Fig. 8 displays he resuls obained in one of he esed siuaions. Thanks o he classificaion algorihm, only measuremens relaed o obsacles in he environmen are aken ino accoun by he probabilisic esimaor. The figure shows he XPF oupu in 5 frames in a global sequence. Two resuling images organized verically are shown for each: The upper one displays he lef frame wih green dos showing he measuremens posiion from he previous classificaion process, and recangles represening he clusering algorihm oupu. Clusers are modelled as cylinders in he 3D world, hus heir image projecion are recangles. The lower image shows he same frame where red dos have been ploed o represen he PF paricles posiion in ha momen. The clusering process has also been used as a final sep of he PF o obain a deerminisic oupu. The resuls displayed in Fig. 6 show ha he racker follows correcly each obsacle posiion in he dynamic and unsrucured indoor environmen. Figure 8. Resuls of he muli-racking algorihm XPF in a real siuaion

6 I can be noiced from he figure, ha he racking algorihm filers emporal occlusions and dual hypohesis for an only objec, and hus, achieving he proposed robusness objecive. The main conclusions of he developed work described in his paper are he following: The es has been run wih a oal number of 600 paricles. As his amoun is kep consan, he XPF execuion ime is also consan, which is a very imporan fac o achieve a real ime developmen. The clusering process increases he likelihood of he new appearing obsacles, improving he robusness ha oher mulimodal esimaors show. The XPF designed can easily handle daa coming up from differen kinds of sensors. This fac makes he final applicaion more flexible han he soluions proposed in he relaed lieraure, based on rigid models for he inpu daa se. Some improvemens o he global muli-racking sysem, which have been menioned wihin he documen, are sill under developmen. In any case, an esimaor o rack a variable and muliple number of objecs has been successfully developed. References: [1] D.B. Reid, An algorihm for racking muliple arges, IEEE Transacions on Auomaic Conrol, Vol. AC-24, nº 6, pp , December [2] M.S. Arulampalam, S. Maskell, N. Gordon, T. Clapp, A uorial on paricle filers for online nonlinear non-gaussian Bayesian racking, IEEE Transacions on Signal Processing, Vol. 50, nº 2, pp: , February [3] A. Douce, J.F.G. de Freias, N.J. Gordon, Sequenial Mone-Carlo Mehods in Pracice, Springer-Verlag, ISBN: , New York, [4] R. Van der Merwe, A. Douce, N. de Freias, E. Wan, The unscened paricle filer, Advances in Neural Informaion Processing Sysems, NIPS13, November [5] S. Thrun, Probabilisic algorihms in roboics, Arificial Inelligence Magazine, winer [6] M. Isard, A. Blake, Condensaion: Condiional densiy propagaion for visual racking, Inernaional Journal of Compuer Vision, Vol. 29, nº 1, pp. 5-28, [7] K. Okuma, A. Taleghani, N. De Freias, J.J. Lile, D.G. Lowe, A boosed paricle filer: muli-arge deecion and racking, Proceedings of he Eighh European Conference on Compuer Vision (ECCV04), Lecure Noes in Compuer Science, ISBN: , Vol. 3021, Par I, pp Prague, May [8] T. Schmi, M. Beez, R. Hanek, S. Buck, Wach heir moves applying probabilisic muliple objec racking o auonomous robo soccer, Proceedings of he Eigheenh Naional Conference on Arificial Inelligence (AAAI02), ISBN: , pp , Edmonon, July [9] J. MacCormick, A, Blake, A probabilisic exclusion principle for racking muliple objecs, Proceedings of he Sevenh IEEE Inernaional Conference on Compuer Vision (ICCV99), Vol. 1, pp , Corfu, Sepember [10] H. Tao, H.S. Sawhney, R. Kumar, A sampling algorihm for racking muliple objecs, Proceedings of he Inernaional Workshop on Vision Algorihms a (ICCV99), Lecure Noes in Compuer Science, ISBN: , Vol. 1883, pp , Corfu, Sepember [11] Y. Bar-Shalom, T. Formann, Tracking and daa associaion (Mahemaics in Science and Engineering, V.182), Academic Press, ISBN: , January [12] C. Rasmussen, G.D. Hager, Probabilisic daa associaion mehods for racking complex visual objecs, IEEE Transacions on Paern Analysis and Machine Inelligence, Vol. 23, nº6, pp: , June [13] M. Marrón, M.A. Soelo, J.C. García, A probabilisic mulimodal algorihm for racking muliple and dynamic objecs, Proceedings of he World Auomaion Congress 2004 (WAC04), ISBN: , Vol. 4, Seville, June [14] M. Marron, M.A. Soelo, J.C. García, Design and applicaions of an exended paricle filer wih a pre-clusering process -XPFCP-, Proceedings of he inernaional IEEE conference Mecharonics & Roboics 2004 (MECHROB04), ISBN: X, Vol. 2/4, pp: , Aachen, Sepember [15] E.B. Koller-Meier, F. Ade, Tracking muliple objecs using a condensaion algorihm, Journal of Roboics and Auonomous Sysems, Vol. 34, pp , February [16] M. Marrón, J.C. García, M.A. Soelo, E.J. Bueno, Clusering mehods for 3D vision daa and is applicaion in a probabilisic esimaor for racking muliple objecs, Proceedings of he Thiry-Firs Annual Conference of he IEEE Indusrial Elecronics Sociey (IECON05), ISBN: , pp: , Raleigh, November 2005.