Moving Target Tracking Based on CamShift Approach and Kalman Filter

Transcription

1 Appl. Math. Inf. Sc. 7 No. 1S pp. 193S-200S (2013) Appled Mathematcs & Informaton Scences An Internatonal 2012 NSP Natural Scences Publshng Cor. Movng arget rackng Based on CamShft Approach and Kalman Flter Kuo-Hsen Hsa 1, Shao-Fan Len 2 and Juhng-Perng Su 3 1 Department of Computer Scence and Informaton Engneerng, Far East Unversty, anan, awan 2 Department of Electrcal Engneerng, Natonal Yunln Unversty of Scence and echnology, Yunln, awan 3 Graduate School of Engneerng, Natonal Yunln Unversty of Scence and echnology, Yunln, awan Correspondng author: Emal: khhsa@cc.feu.edu.tw Receved Nov. 25, 2011; Revsed Jan. 8, 2012; Accepted Jan. 31, 2012 Publshed onlne: 2012 Abstract: he survellance system has been developed for decades. It has reduced the crme and protected the lves and propertes of people successfully. Some survellance systems are composed of PZ cameras. herefore the movng people or target objects could be tracked by the survellance system. he survellance system becomes more and more nterment. However, there are two problems occurred, target recognton and target shelter. In ths paper, we proposed the method of trackng a movng object for mprovng the performance of ntellgent survellance system. he method combned CamShft and Kalman flter for trackng the movng object n a complex background or n an occluson case. he proposed method s very effcent and that s easly mplemented n a real-tme system. hree major problems of survellance system desgn, movng object detectng, movng object trackng and trackng the object n occluson, have been conquered n the expermental results. Keywords: CamShft, Kalman flter, HSV color model, Image trackng. 1 Introducton he survellance system has been developed for many years. he survellance systems are usually used for preventng crmes and that also protected the lves and propertes of people successfully. In recent years, the fast development of CCD chp and computer scence provdes the good bass for multmeda vdeo survellance system. oday, the survellance system s an mportant part for securty and vglance tasks. At present, the PZ camera s very popular for survellance system due to the Pan-lt-Zoom motons [1]. he PZ camera could be regarded as a servo system wth two-axs moton [2]. In prevous researches, many algorthms or control schemes have been proposed for makng the survellance system more ntellgently and autonomously. Al Haj, Bagdanov, González and Roca proposed recursve Bayesan lter framework for estmatng the camera and world parameters [3]. In the desgn of ntellgent survellance systems, two mportant problems, target recognton problem and target shelter problem, have to be consdered. here are usually several movng targets n one frame. herefore, developng an excellent method for recognzng and trackng the dubous movng target s especally mportant. In addton, the movng targets are often obstructed by other objects, thus how to produce the object movng trajectory for PZ camera trackng s another dffcult task. In ths paper, three problems for ntellgent survellance system desgn wll be dscussed. hey are (1) movng object detectng, (2) movng object trackng and (3) trackng the object n occluson. Image processng technology promoted the development of ntellgent survellance system. In prevous researches, the bnary segmentaton and background subtracton method s used for detectng movng objects n [4] and [5],

2 194 Kuo-Hsen Hsa et al.: Movng arget rackng Based on CamShft Approach respectvely. Such methods are very effectve for segmentng the movng object. However, when the background s changed a lot or the objects are n occluson, the methods n [4] and [5] are not easy to detect movng objects. herefore, the 3D nformaton from PZ camera s used for detectng movng objects, and the ssue of background changed has been overcome. But the problem of object n occluson s stll not mentoned. Hartoglu [6] proposed the slhouette projecton method for fndng out the movng people n occluson. Wang [7] used the feature matchng and kernel trackng for predctng the targets. We wll propose n ths paper the method whch combnes D (emporal Dfferencng) [8], CamShft (Contnuously Adaptve Mean Shf [9] and Kalman flter for solvng the problems smultaneously. D can detect the movng object very effectvely, but s unfavorable for trackng for background changng a lot. herefore, CamShft s utlzed for trackng the target. CamShft s the template matchng method n a sequence of mages. he template of target s establshed by the color model HSV (hue, saturaton, value) [10]. CamShft can easly catch the movng target. Kalman flter provdes an optmal estmaton for lnear systems. herefore, t s added to estmate the movng target n occluson. In the followng sectons, the proposed method wll be dscussed. In the expermental results, some smulaton cases wll be used for demonstratng our method. Prevous Image Present Image D Image Pre-proccessng Calculate the model of color densty dstrbuton Establsh the template Camshft Kalman flter No Movng object Lock the target Fgure 1: Flowchart of the proposed methods. Yes he camera system for our applcaton s Sony D70 PZ camera. he capture range of the camera s llustrated as fgure 2. here are many features of D70 PZ camera, such as: 2 Movng arget rackng Method A. Detectng the movng target n mage sequence he proposed method s llustrated as fgure 1. here are three major parts n ths method. he frst s detectng and segmentng the movng target. he second part s template computaton. Last, the CamShft and Kalman flter are used for trackng and lockng the movng target. Commonly used methods n movng target detecton are BS (Background Subtracton) [8], D (emporal Dfferencng) and OF (Optcal Flow). he computaton tme of OF s larger than BS and D. hus OF s not sutable for a real-tme system. he background template s not easy to be computed wth BS n changng background. Hence D s used for detectng the movng object n our applcaton. (a) Sde vew (b) op vew Fgure 2: Capture range of Sony D70 PZ camera [11]

3 Kuo-Hsen Hsa et al.: Movng arget rackng Based on CamShft Approach Wde Range Pan/lt: 340 degrees Pan / 120 degrees lt. 2. Excellent Low Lght Senstvty: Mnmum llumnaton less than 1 Lux mm mount for optonal converson lens. 4. Fully Remote Controllable va RS-232C (VISCA) or RS-422. It can be seen that D70 s an excellent devce for survellance system. B. HSV Color Space HSV (hue, saturaton, value) s a very useful cylndrcal-coordnate representaton of ponts n an RGB color model. he HSV color model descrbes more color features of object than RGB model. Fgure 3 llustrates the HSV color model. Cheng proposed some applcaton examples of mean shft [13]. he most representatve paper s proposed by Comancu, Ramesh and Meer [14]. In [15], mean shft s appled to the analyss of the feature space. In summary, mean shft algorthm s a method for fndng the maxmum and the most concentrated value of the color probablty dstrbuton. In addton, mean shft approach appled to trackng ssue has been verfed by other researchers. In the followng t s a bref descrpton of mean shft trackng. Assume the target template s {x } =1,2 n n R d. he multvarate kernel densty of pont X 0, ( X ) s qˆ u 0 n 2 1 x0 x qˆ u ( X 0 ) k (4) n h 1 h where k(x) s defned as 1/ d 1 2 k( x) (2 ) exp( x ) (5) 2 and h s the wndow wdth. Moreover, the multvarate kernel densty of the canddate movng object at poston Y s Fgure 3: HSV model [10]. Snce the HSV color model s not easy to be mplemented n computer graphcs, the HSV color model usually transformed nto RGB color model, practcally. he conversonal relatonshp between RGB and HSV are V max( R, G, B) (1) V mn R, G, B S (2) V G B60/ S, f V R H 180 B R60/ S, f V G (3) 240 R G60/ S, f V B Snce the advantage of HSV s the property of not senstve to the change of lght, the template wll be establshed by HSV. C. CamShft he dea of mean shft s frst proposed by Fukunaga and Hostetler n 1975 [11]. Mean shft s utlzed for estmatng the gradent vector of PDF (Probablty Densty Functon) n [12]. In 1995, n 1 Y x pˆ u ( Y ) k( ) (6) n h h 1 herefore, searchng and trackng ssue can be regarded as the smallest dfference between the values of the templates. he matchng cost functon s the Bhattacharyya coeffcent [16]: ρ( y) ρ[ p( u, y), q( u)] p( u, y) q( u) (7) m u1 where p(u, y) s the u th color ndex of the canddate movng object, q(u) s the u th color ndex of the template. Next, the movng vector X s X K( x x) w( x) ( x K( x x) w( x) 2 x). (8) he X s converged by teratons. In (5), the w(x) ndcated the correspondence between the template and search block at pxel x.

4 196 Kuo-Hsen Hsa et al.: Movng arget rackng Based on CamShft Approach w( x) q( I( x))/ p( I( x)) (9) where q(i(x)) and p(i(x)) are the color dstrbutons of templates and the search block, respectvely. he K(.) s usually a symmetrc kernel functon. he steps of mean shft trackng algorthm are Step 1: Set the regon of nterest (ROI) of the nput mage. Step 2: Calculate the color hstogram of the target mage as the template. Step 3: Calculate the target's color hstogram of next mage and start to search. Step 4: Calculate the movng vector X. (If X s dvergent, go back to Step 2.) Step 5: Compute the Bhattacharyya coeffcent for fndng the best matchng block. CamShft algorthm mproves the searchng effcency of mean shft trackng algorthm. From Step 5 of mean shft trackng algorthm, the coordnate of the best matchng block to be the ntal searchng poston at next contnuous mage s recorded. he procedure of CamShft algorthm s llustrated as fgure 4. Input mage and ROI Intalzaton Calculate the color hstogram of ROI (color hstogram of target ) ransform the color hstogram of target nto probablty dstrbuton Next mage nput and update the ROI Compute the sze and poston of ROI Compute the Centrod of Mean Shft(MS) Set the center of searchng wndow to the Centrod of Mean Shft Convergence? Yes Regulate the sze of ROI New ROI Fgure 4: CamShft algorthm. D. Kalman flter Kalman flter [17][18] s a well-known method n engneerng applcatons. It s a tme-doman flter n whch estmaton of states s performed by two procedures, namely, predcton (tme update) and correcton (measurement update). State estmaton s acheved by weghtng sum of state predcton and output measurement. In our applcaton, the Kalman flter s utlzed for No estmatng states of the movng target n occluson. he process of Kalman flter s llustrated n fgure 5. me Update (1) Predct the state ˆ( ) ( 1) ~ ahead s t Φ t s ( t 1) (2) Predct the error covarance ahead ˆ ~ Φ( t 1) t 1) Φ( t 1) Q( t 1) Intal estmaton for Measurement Update (1) Compute the Kalman gan K( Pˆ( Pˆ( R( 1 (2) Update estmaton wth measurement C( ~ s ( sˆ( K( C( ˆ( s t ) (3) Update the error covarance ~ I K( Pˆ( t ) ŝ and Fgure 5: he process of Kalman flter. Assume a Newton s system wth measurement s descrbed as Systemstate: s( Φ( t 1) s( t 1) w( (10) Measurement : C( s( v( where Φ s the state transton matrx, H s measurement matrx, w( and v( s the nose. he two major steps of Kalman flter are Predcton and Correcton. Predcton (me update): sˆ( Φ( t 1) ~ s ( t 1) ~ Pˆ( Φ( t 1) t 1) Φ( t 1) Correcton (Measurement update): K( Pˆ( ~ s ( ~ sˆ( K( C( I K( Pˆ( pˆ Pˆ( ˆ( s (11) Q( t 1) R( 1 (12) where ŝ and s ~ are a pror estmate and posteror estmate, respectvely; Pˆ and P ~ are a pror

5 Kuo-Hsen Hsa et al.: Movng arget rackng Based on CamShft Approach 197 estmate error covarance and posteror estmate error covarance, respectvely; K s the Kalman gan and C s the measurement vector; R and Q are the measurement error covarance and process nose covarance. he Predcton and Correcton are cyclng alternately. E. Movng model of target object Before estmatng the movng of object by Kalman flter, the movng model needs to be establshed. he movng objects, such as movng people or stream of vehcles are usually nerta Newton's systems. herefore, the states of movng objects n mage sequence can be establshed. he most mportant states of Newton's system are poston, velocty and acceleraton. In other words, we wll establsh the second-order model of target object for Kalman flter calculatng. From CamShft, the poston of the target object n the mage sequence P t s and fgure 6(c-2) are the target segmentaton results of fgure 6(b-1) and fgure 6(c-1), respectvely. Moreover, for demonstratng the recognton of the algorthm, we added some objects whch are of smlar color of the target. Fgure 6(c-1) and fgure 6(c-2) show that the CamShft s very robust. In ths case, the mage sze s and the frame rate s 26 fps. (a) P P P P (13) t t1 t1 t2 herefore, the velocty and acceleraton could be obtaned by every capturng nterval. By usng (10) and (13), the state space of Kalman flter s (b-1) (b-2) s( Φ( t 1) s( t 1) w( Pt 0 Pt 2 w( 0 1 t ) (14) (c-1) (c-2) Pt 1 where s ( t 1) and Φ ( 2 1. Pt Moreover, the parameters of the measurement Pt n (10) are defned as s (, H ( 1 0 Pt 1 and C ( P t. By ths way, the Kalman flter can estmate the movng object contnuously. 3 Expermental Results A. CamShft trackng he color converson model used n ths paper s HSV. We used a small dskette-case of lght green color to be a target and fgure 6 demonstrated the trackng results. In fgure 6(a), the color hstogram of target template s concentrated n green. In fgure 6(b-1) and fgure 6(c-1), we test the algorthm when target movng n complex envronment wth lght background. Fgure 6(b-2) (d-1) (d-2) Fgure 6: he trackng results of CamShft. We captured the fragments from the vdeo of trackng dskette-case. he contnuous poston of target has been computed by CamShft. Fgure 7(a) and fgure 7(b) show the results estmated by Kalman flter from frame 42 to 107 and frame 206 to 271, respectvely. he MSE of the estmatons n fgure 7(a) and fgure 7(b) are and 8.234, respectvely. he unts of x-axs and y-axs n fgure 7 are pxels. B. rackng by assstance of Kalman flter

6 198 Kuo-Hsen Hsa et al.: Movng arget rackng Based on CamShft Approach he Kalman flter provdes an acceptable estmaton poston of target when the target occluded temporarly. In fgure 8, t s the case that the movng person s occluded by another one. From the smulaton result, t shows that the trackng result of the method s well. It does not only track the specfcally movng person, but also provde the estmaton target postons whle n occluson. In the followng, descrptons about the trackng and estmatng procedures are stated. estmated postons made by Kalman flter. When the target appears agan, the CamShft wll catch the target mmedately based on the estmated postons. he mage sze s he frame rate s 15 fps. (1) (5) (2) (6) (a) (3) (7) (4) (8) Fgure 8: Movng person trackng results. 4 Conclusons (b) Fgure 7: Poston estmatng results of a movng dskette-case by Kalman flter. Frstly, the CamShft catches the person wearng a red -shrt. Next, he s occluded by the person wearng gray jacket. he Kalman flter can provde the estmated postons contnuously. Fnally the estmaton poston wll be the ntal poston of CamShft when the target appears agan. In ths case, the target s occluded n 8 frames, and the Kalman flter provdes the possble postons. he square marks n fgures 8(4) (6) are the he PZ camera s a very popular devce for survellance systems. he survellance systems become more and more ntellgent for human lfe. In ths paper, we focused on the problem of movng object trackng wth the object n occluson. We proposed a smple and effcent method. hs method s sutable especally for survellance system desgn. he method can detect the movng objects by D ntally and establsh the target template n HSV color space for CamShft trackng. Fnally, the Kalman flter provdes the estmated postons when the target s n occluson. In expermental results, we demostrated that the green CD case could be tracked by CamShft. he

7 Kuo-Hsen Hsa et al.: Movng arget rackng Based on CamShft Approach 199 CamShft trackng performance s very good and very robust. In addton, we also showed the case of the movng people passed behnd another one. he method can not only track the moton but also estmate the trajectory of the movng people successfully. herefore, n ths paper, three major problems, movng object detectng, movng object trackng and trackng the object n occluson, have been conquered successfully by our method. References [1] C.S. Yang, R.H. Chen, C.Y. Lee and S.J. Ln, PZ camera based poston trackng n IP survellance system. In Proc. of the 3rd Int l Conference on Sensng echnology. (2008), [2] M. Al Haj, A.D. Bagdanov, J. González and F.X. Roca, Reactve object trackng wth a sngle PZ camera. In Proc. of the Int l Conference on Pattern Recognton (ICPR 20). (2010), [3]. Wang, G. Lu and W. Xe, Vsual servong control of vdeo trackng system for trackng a flyng target. In Proc. of the Int l Conference on Advanced Intellgent Mechatroncs. (2011), [4] D.V. hombre, J.H. Nrmal and D. Lekha, Human detecton and trackng usng mage segmentaton and Kalman flter. In Proc. of the Int l Conference on Intellgent Agent & Mult-Agent Systems. (2009), 1-5. [5] Z. Xu and H. R. Wu, Smart vdeo survellance system. In Proc. of the IEEE Int l Conference on Industral echnology (ICI 2010). (2010), [6] I. Hartaoglu, D. Harwood, and L.S. Davs, W 4 : real-tme survellance of people and ther actvtes. IEEE ransactons on Pattern Analyss and Machne Intellgence. (2000), 22(8), [7] G.L. Wang, D.Q. Lang, Y.C. Wang and Z.H. Hu, Algorthm for trackng of fast moton objects wth adaptve mean shft. In Proc. of the Eghth ACIS Int l Conference on Software Engneerng, Artfcal Intellgence, Networkng, and Parallel/Dstrbuted Computng. (2007), [8] B. Shoushtaran, and H.E. Bez, A practcal adaptve approach for dynamc background subtracton usng an nvarant colour model and object trackng. Pattern Recognton Letters. (2005), 26(1), Sydney, Australa, ACS. (2004), 36, 3-7. [10] R.C. Gonzalez and R.E. Woods, Dgtal Image Processng, 2nd ed. Prentce Hall Press, (2002), 295. [11] Sony EVID 70P, EVID70P/. [12] K. Fukunaga, L.D. Hostetler, he estmaton of the gradent of a densty functon, wth applcatons n pattern recognton. IEEE ransactons on Informaton heory. (1975), 21(1), [13] Y. Cheng, Mean shft, mode seekng, and clusterng. IEEE ransactons on Pattern Analyss and Machne Intellgence. (1995), 17(8), [14] D. Comancu, V. Ramesh and P. Meer, Real-tme trackng of non-rgd objects usng mean shft. In Proc. of the IEEE Conference on Computer Vson and Pattern Recognton. (2000), 2, [15] M.Y. Ju, C.S. Ouyang and H.S. Chang, Mean shft trackng usng fuzzy color hstogram. In Proc. of the Int l Conference on Machne Learnng and Cybernetcs (ICMLC). (2010), 6, [16] A. Bhattacharyya, On a measure of dvergence between two statstcal populatons defned by ther probablty dstrbutons. Bulletn of the Calcutta Mathematcal Socety. (1943), 35, [17] R.E. Kalman, A new approach to lnear flterng and predcton problems. ransactons of the ASME Journal of Basc Engneerng. (1960), 82 (D), [18] R.S. Bucy and P.D. Joseph, Flterng for Stochastc Processes wth Applcatons to Gudance, 2nd ed. AMS Chelsea Publ., (2005). Kuo-Hsen Hsa receved the Ph.D. degree n electrcal engneerng from the Natonal Sun Yat-Sen Unversty, awan, n He s currently an assocate professor of the Department of Computer Scence and Informaton Engneerng, Far-East Unversty, awan. Hs research nterests are n the area of dfferental game theory, fuzzy set theory, mage-assstant measurement and control software desgn. [9] J.G. Allen, R.Y.D. Xu, J.S. Jn, Object trackng usng CamShft algorthm and multple quantzed feature spaces. In Proc. of the Pan-Sydney Area Workshop on Vsual Informaton Processng (VIP2003), Conferences n Research and Practce n Informaton echnology,

8 200 Kuo-Hsen Hsa et al.: Movng arget rackng Based on CamShft Approach Shao-Fan Len was born n awan, R.O.C., on February 13, He receved the B.S. degree from Department of Electronc Engneerng of Huafan Unversty n 2004 and receved the M.S. degree from ungnan Unversty n He s a Ph.D. student of Graduate School of Engneerng Scence and echnology of Natonal Yunln Unversty of Scence and echnology from 2006 to today. Hs research nterests nclude computer vson and vson servong system control. Juhng-Perng Su was born n awan n1958. He receved the M.S. degree n electrcal engneerng from the Natonal awan Unversty of Scence and echnology, ape, awan, n 1986, and the Ph.D. degree n electrcal engneerng from Sun Yat- Sen Unversty, Kaohsung, awan, n Snce 1991, he has been wth the Natonal Yunln Unversty of Scence and echnology, Yunln, awan, where he s currently a Professor n the Department of Electrcal Engneerng. Snce 2004, he has been conductng research projects on autonomous unmanned helcopters. Hs current research nterests nclude the desgn of autonomous unmanned vehcles, robust estmaton and optmal control, ntellgent systems, and nonlnear control. Dr. Su was a recpent of the dstngushed research award from Yunln Unversty of Scence and echnology n 2005 and awards from varous domestc compettons.