Kalman Filtering Based Object Tracking in Surveillance Video System

Transcription

1 ( ) Proceeding of the 3rd (2011) CUSE International Conference Kalman Filtering Baed Object racking in Surveillance Video Sytem W.L. Khong, W.Y. Kow, H.. an, H.P. Yoong, K..K. eo Modelling, Simulation and Computing Laboratory School of Engineering and Information echnology Univeriti Malayia Sabah Kota Kinabalu, Malayia Abtract In the field of motion etimation for urveillance video, variou technique have been applied. One of the common approache i Kalman filtering technique and it i intereting to explore the extenion of thi technique for the prediction and etimation of motion via the image equence. In thi paper, a moving object tracking in urveillance video uing Kalman filter i propoed. he typical Kalman filter i good in tracking the poition of a moving object. However, when dealing with occluion, the typical Kalman filter i not able to keep tracking and predicting the poition of the occluded moving object. During occluion, the information of moving object i not available for detection and tracking. he lacking of occluion cene determination and prediction ability caue the exiting Kalman filter fail in tracking occluded object. Beide that, in the cae of tracking multiple moving object, exiting Kalman filter will experience difficultie to identify the repective object. herefore, in order to encounter thee problem, an object tracking method uing enhanced Kalman filter will be developed. he ability of tracking occluded moving object will be added to increae the efficiency during tracking. Furthermore, object recognition feature will be added too to increae the accuracy of the object tracking ytem. Keyword - Kalman filter; object tracking; urveillance ytem I. INRODUCION he reearche for egmenting, etimating, and moving object tracking in video have received great attention for the lat few year. he moving object tracking i an important iue in video ytem, uch a urveillance, port reporting, video annotation, and traffic management ytem. Difficultie in tracking object can arie due to the abrupt object motion, changing appearance pattern of the object and the cene, non-rigid object tructure, object-to-object and object-to-cene occluion. racking i uually performed in the context of higher-level application that require the location or hape of the object in every frame. he typical Kalman filter ha faced the problem in tracking during occluion. A during occluion, the information of moving object i not available for detection and tracking. he lacking of occluion cene determination and prediction ability make the typical Kalman filter fail in tracking the occluded object. Beide that, in the cae of tracking multiple moving object, typical Kalman filter will experience difficultie to identify the repective object. herefore, the typical Kalman filter ha weakne when dealing with occluded moving object. In order to encounter thi problem, an object tracking method uing enhanced Kalman filter with the ability of tracking occluded object and object recognition feature will be developed to increae the efficiency and accuracy of the object tracking ytem. II. REVIEW OF OBJEC RACKING MEHODS Object tracking i an important tak within the field of computer viion. here are three key tep in video analyi: intereting moving object detection, tracking of uch object from frame to frame, and analyi of object track to recognize their behavior. herefore, the ue of object tracking may be utilized in the tak uch a motion-baed recognition, traffic monitoring, automated urveillance, a well a vehicle navigation. he complexity of object tracking i due to the noie in image, cene illumination change, complex object motion, and partial and fully object occluion. Mot of the tracking algorithm aume that the moving object i move in mooth and no udden change. emplate are imple geometric hape or ilhouette [1]. For object whoe poe doe not vary much during tracking are uitable to ue template method. Right feature election play a critical role in tracking. Feature election i important in object repreentation. For example, color i ued a a feature for hitogram-baed appearance repreentation. he RGB (red, green, blue) color pace i uually ued to repreent color in image proceing. he difference of color in the RGB pace do not linearly repone to the difference of color perceived by human. herefore, a variety of color

2 ( ) Proceeding of the 3rd (2011) CUSE International Conference pace have been ued in tracking due to thi inefficient matter. Edge detection i ued to identify trong change in image intenitie of the object boundarie. An important property of edge i that they are le enitive to illumination change compared to color feature. Canny Edge detector [2] i the mot popular edge detection method. he interet point in image are found uing point detector which have an expreive texture. A commonly ued interet point detector include Moravec interet operator [3], Harri interet point detector [4], KL detector [5], and SIF detector [6]. A background image i modeled and finding the difference with the correpondence in order to find the tracking object. A moving object i categorized with an obviou change in an image region from the background image. Point tracking method ha two main categorie, determinitic method and tatitical method. he determinitic method ue qualitative motion heuritic [7] to contrain the correpondence problem while the tatitical method explicitly take the object meaurement and take uncertaintie into account to etablih correpondence. he example of point tracking method include Kalman filter and particle filter. Kernel i the object hape and appearance. he motion of the moving object from one frame to the next frame i computed in Kernel tracking. he object motion i generally in the form of parametric motion (uch a tranlation, conformal, and affine) or the dene flow field computed in ubequent frame. III. KALMAN FILER ECHNIQUE IN OBJEC RACKING he difference of the typical Kalman filter and the enhanced Kalman filter are highlighted by the involvement of occluion cene determination and occluion rate calculation which making the tracking during occluion i capable. he occluion rate feature will be activated once the detection howing that the moving object i fully occluded. Beide that, object recognition capability i alo added to the enhanced Kalman filter o that the tracking target can be recognized from other. hi feature i running all the time to aure that the targeted object will be recognized all the time. A. he ypical Kalman Filter he Kalman filter i a recurive etimator. hi mean that only the etimated tate from the previou time tep and the current meaurement are needed to compute the etimate for the current tate. hu, no hitory of obervation or etimate i required. he Kalman filter ha two ditinctive feature. One i that it mathematical model i decribed in term of tate-pace concept. Another i that the olution i computed recurively. Uually, the Kalman filter i decribed by ytem tate model and meaurement model. he tate-pace model i decribed a ytem tate model and meaurement model a hown in (1) and (2) repectively. ( = O( t 1) ( t 1) w( (1) z ( = H ( ( v( (2) Where O ( t 1) and H ( are the tate tranition matrix and meaurement matrix repectively. he w ( and v( are white Gauian noie with zero mean. Kalman filter have two phae: prediction tep and correction tep. he prediction tep i reponible for projecting forward the current tate, obtaining a prior etimate of the tate (. he tak of the correction tep i for the feedback. It incorporate an actual meaurement into the prior etimate to obtain an improved poterior etimate (, which i written a hown in (3). ( ( k( [ z( H ( ( ] = (3) Where k ( i the weighting and i decribed a hown in (4). p k( = ( H ( [ H ( p ( H ( R( ] p ( H ( H ( p ( H ( = p In (4), the ( R( 1 (4) i the priori etimate error covariance. It i defined a hown in (5). p ( = E[ e ( e ( ] (5) Where e ( = ( ( i the prior etimate error. In addition, the poteriori etimate error covariance i defined a hown in (6). p ( = E[ e ( e ( ] (6) Where e ( = ( ( i the poteriori etimate error. he prediction tep and correction tep are executed recurively in the definition a hown in (7), (8), (9), (10) and (11). Prediction tep: ( = O( t 1) ( t 1) (7) p ( = O( t 1) p ( t 1) O( t 1) Q( t 1) Correction tep: k( = p ( H ( [ H ( p ( H ( R( ] ( ( k( [ z( H ( ( ] 1 (8) (9) = (10)

3 ( ) Proceeding of the 3rd (2011) CUSE International Conference p ( [1 k( H ( ] p ( = (11) he prediction-correction cycle i repeated. Looking at (9), the meaurement error R( and Kalman gain k( are in invere ratio. he maller the R (, the gain k ( weight more heavily. In thi cae, the meaurement i more truted, while the predicted reult i le truted. However, a the a priori etimate error p ( approache zero, the gain k ( weight the reidual le heavily. he actual meaurement i truted le and le, while the predicted reult i truted more and more. B. he Enhanced Kalman Filter Baically, the algorithm for the typical Kalman filter and the enhanced Kalman filter would be the ame jut that an occluion rate i added into the correction tep once the moving object detection in conecutive frame calculated i zero, that i, the moving object i being occluded. Once the moving object i being occluded, it conecutive predicted poition will be relying on the latet lat two conecutive frame which are ued to calculate the rate of occluion. After the rate of occluion i gained, the next poition of the occluded moving object will be predicted until the object i moving put from the occluded area. he occluion rate can be defined a in (12). OcclRate = x( t 1) δ ( (12) Beide that, object recognition feature i alo added into the enhanced Kalman filter o that other moving object will be eliminated from the targeting moving object during tracking. hi feature i ueful for tracking condition that ha multiple moving object. he Kalman filter provide an efficient way to etimate the tate of a linear proce and it minimize the mean of the quared etimation error. Beide that, the Kalman filter i a recurive etimator. hi mean that only the etimated tate from the previou time tep and the current meaurement are needed to compute the etimate for the current tate. No hitory of obervation or etimate i required. IV. YPICAL KALMAN FILER (KF) IN OBJEC RACKING AND ANALYSIS In thi ection, analyi of object flow uing typical Kalman filter (KF) will be hown. he typical KF capability will be teted out under ome moving object condition uch a moving in contant direction and velocity, variable ize and hape, and long lating occluion. he reult of object flow without occluion will be hown. Some of the reult are teted uing recorded video and ome of them are uing animation for the eae of the condition that video recording i difficult to perform. A. Contant Direction and Velocity From Fig. 1, the typical KF uccefully tracked the moving object in every ingle frame. he green circle howing that the moving object i tracked and plotted effectively. A a concluion, the typical KF technique i uitable in tracking moving object in contant direction and velocity. B. Variable Size and Shape From Fig. 2, the typical Kalman filter i able to predict and track during the experiment. he firt ize and hape changing i occurred from frame 4th to frame 6th, however, the typical KF facing no problem in continuing tracking the moving object. Figure 1. he experimental reult of moving object with contant velocity Figure 2. he experimental reult of moving object with variable ize and hape C. Sudden Change in Velocity From Fig. 3, from the beginning until 4th frame, the object wa moved in contant velocity and uddenly changed to a fat velocity in 5th frame. At the moment, the typical KF tracked the poition that i a bit deviated from the real poition, a pointed by the black arrow. However, until the 7th frame, the green circle fat getting back to the real poition and keep tracking the moving object until the end. hu, the typical Kalman filtering method i not o uitable for tracking moving object changing velocity uddenly. However, it will get back to the next true poition in a very hort time.

4 ( ) Proceeding of the 3rd (2011) CUSE International Conference Figure 3. he experimental reult of moving object changing direction uddenly D. Long Lating Occluion (KF) From Fig. 4, there wa a table that the moving object will pa through behind it o that a long lating occluion cene wa created. At frame 13th, the moving object tarted to pa through from the behind of the table. he moving object wa being occluded by the table from frame 17th until frame 29th. During that time, tracking could not be performed and thu no reult were hown in the total 13 frame. Finally, at frame 30th, the moving object tarted to come out from behind of the table, however, the tracking poition i far away from the poition where it hould be, a pointed by the black arrow. he ame phenomenon perited at frame 31t. Started from frame 32nd until frame 34th, the green circle lowly going back to it right poition and continued it tracking mechanim. hat i, a a concluion, the typical KF technique i not uitable in tracking moving object with long lating occluion. On the other hand, it i till able to track back the target lowly once the moving object wa not been occluded. E. Occluion of wo Moving Object (KF) From Fig. 5, the targeted moving object i the baketball (brown color moving objec a can be een at the frame 11th. At firt, the typical KF worked well without loing it targeted moving object until frame 21t. However, at frame 2nd, the typical KF tracked the wrong moving object, the football (white color moving objec intead of tracking baketball, a can be een from the pointed black arrow. he ame mitake repeated in the 23rd frame, 26th frame, 27th frame, and 42nd frame. hi i not a good phenomenon a the typical KF alway howed a fale tracking. Until frame 43rd, the two moving object getting cloer and it i the time for the typical KF to filter out the unwanted object. Unfortunately, at frame 58th, the reult i not a expected a the green circle tracking the wrong object until the end of the experiment. A a concluion, the typical Kalman filter without object recognition capability i not uitable in tracking the occluion of two moving object. Figure 4. he experimental reult of moving object with long lating occluion (KF) Figure 5. he experimental reult of occluion of two moving object (KF) F. Fale racking After Occluion From Fig. 6, from the beginning until the 6th frame, there wa nothing wrong with the tracking of the typical KF. At frame 6th, the yellow moving object tarted to enter behind of the block. he yellow moving object wa being occluded from frame 7th until frame 16th. From the 17th frame until the 20th frame, another red moving object i coming out from the block intead of the initial yellow object. he typical KF keep tracking the red moving object a thi wa the only object that can be tracked. At frame 22nd, the initial yellow moving object finally moved out from the block but till the typical KF tracked the wrong object a can be een from the black pointed arrow. hi mitake keep repeated until the end of the experiment. From the reult, the typical KF

5 ( ) Proceeding of the 3rd (2011) CUSE International Conference without object recognition ability i not uitable in tracking occluded object where there i other moving object added in to confue the tracking mechanim. until the end of tracking. hu, the EKF with occluion rate feature i able to track the moving object even during occluion. Figure 6. he experimental reult of fale tracking after occluion (KF) V. ENHANCED KALMAN FILER (EKF) IN OBJEC RACKING AND ANALYSIS In thi analyi ection, thoe reult which are not giving a well and atified reult uing typical KF will be teted again uing the EKF. Mot of the reult that are not giving a nice tracking performance are related to occluion condition. hu, the main focu in thi chapter i tried to overcome the occluion problem that are facing in the previou ection. he occluion problem that i facing in the previou chapter will be overcome with the occluion cene calculation feature that i added into the EKF, the moving object poition that i being occluded will be predicted and plotted out. Beide that, ome cae that are confue in recognizing the targeted object will be teted under thi ection too. he object recognizing feature that i added into the EKF for thi paper i object ize recognition. A. Long Lating Occluion (EKF) From Fig. 7, from the beginning of the tracking until the 12th frame, EKF performed well without loing the target. Starting from frame 16th, the moving object (football) entered behind the block and diappeared totally from frame 16th till frame 28th. During thi period, the EKF i till able to predict the poition of the moving object a indicated by the black arrow. At frame 29th, the moving object tarted to come out from the occluion cene, and the EKF tracked the moving object immediately. Comparing to Fig. 4, the ame cae but uing typical KF, the EKF can capture back it targeted object a oon a it moved out from the occluion cene. hi i much better than the typical KF a the typical KF need longer time to track back it target. After the occluion cene, the EKF work well Figure 7. he experimental reult of moving object with long lating occluion (EKF) B. Occluion of wo Moving Object (EKF) From Fig. 8, the EKF performed well from the beginning until the 43rd frame, which i before the two moving object occluded. Beide that, at frame 16th, the targeted moving object that the EKF wa going to track i the brown moving object (baketball) intead of the white moving object (football). At the 44th frame, indicated by the black arrow, at the moment the two moving object croed together, the EKF recognized which moving object wa the targeted object from the beginning. he recognizing mechanim wa performed throughout the whole tracking ytem. Finally, at frame 58th, the EKF uccefully recognizing the brown moving object and the tracking mechanim continued till the end. hi wa a much better reult comparing to Figure 5.7, the ame cae but uing typical KF, the EKF did not face the problem of loing it target during occluion. Furthermore, the cae uing EKF did not confue it targeted object throughout the whole tracking ytem, which the typical KF failed to perform a can be een from Fig. 5. A a concluion, the EKF with object recognizing and occluion feature would not loe it targeted moving object throughout the whole tracking period even during occluion. C. Occluion with Object Recognition (EKF) From Fig. 9, the EKF performed well from the beginning until the 6th frame before it entered the blue color block. Started from frame 7th, the moving object moved to behind of the block and thi ituation perited until frame 16th. During the occluion period, the EKF kept predicting the poition of every ingle frame without loing them. For example at frame

6 ( ) Proceeding of the 3rd (2011) CUSE International Conference 11th, the predicting poition of the moving object i at around the centre of the block. Looking at frame 17th, there wa another red quare object coming out firt intead the original yellow object, a indicated by the black arrow. At firt, the EKF tracked the red quare object a it i the only object moving in the frame. Until frame 22nd, the original yellow moving object coming out and EKF recognized it and tracked it immediately a pointed by the black arrow. D. Dicuion From all of the experimental reult hown above, the EKF perform well in the condition that ha occluion and require object recognition feature. he enhanced part in the EKF compare to the typical KF i that the EKF ha added occluion for the condition that the moving object i being occluded. Beide that, object recognition feature added into the EKF will definitely increae the accuracy during tracking. he tracking peed of the EKF i highlighted. With the ability of tracking occluded object, once the object moving out from the occluion cene, the EKF can immediately track the object almot with no delay a can be een from the above reult. hi i a good improvement compare to the typical KF which need more frame o that it can capture back it target. he addition of object recognition feature i alo another improvement that help the EKF to recognize it target without confuing. he previou experimental reult howed that the typical KF fail in recognizing targeted object. VI. CONCLUSION AND FUURE WORK Fig. 8. he experimental reult of occluion of two moving object (EKF) A. Concluion Object tracking uing Kalman filter ha been propoed in thi paper. he typical Kalman filtering method uccefully track the moving object poition in ome kind of ituation uch a the object keep changing hape and ize, lightly occluion, and move in contant direction and velocity. However, when dealing with the moving object with occluion and require object recognition feature to eliminate the confuion between multiple object, the typical Kalman filter doe not give a atified and accurate reult. Difficultie in tracking object can arie due to abrupt object motion, changing appearance pattern of the object and the cene, non-rigid object tructure, object-to-object and object-to-cene occluion, and camera motion. In thi project, the main difficulty that I am facing i that the typical Kalman filter i not uitable to implement in a tracking ytem where the moving object i being occluded. Figure 9. he experimental reult of occluion with object recognition (EKF) hen, from frame 22nd onward till the end, the EKF keep tracking the yellow moving object without confuing it with another moving object. hi i a much better reult comparing to Fig. 6, the ame cae but uing typical KF, the EKF could recognize it targeted object once another object wa trying to confued it. he typical KF failed in recognizing it target. A a concluion, the EKF with object recognizing and occluion feature ha the capability to recognize it targeted moving object. he performance i much better than the typical KF. hu, the enhanced Kalman filter i ued to overcome the matter faced by the typical Kalman filter. he enhanced Kalman filter ha added in occluion cene handling and occluion rate calculation a well a object recognition feature to make up the weaknee of the typical Kalman filter. In concluion, the enhanced Kalman filter ha make up the weaknee faced by the typical Kalman filter and the reult are encouraging alo. Obviouly the performance ha been improved and the tracking reult are more accurate and precie. B. Future Work he preented enhanced Kalman filter i not perfect enough a it i not capable to track moving object during occluion with a pattern of motion uch a inuoidal motion

7 ( ) Proceeding of the 3rd (2011) CUSE International Conference or zigzag motion. he enhanced Kalman filter preented in thi paper can only predict the poition of the occluded moving object with fixed direction and velocity. hu, in future work, perhap the enhanced Kalman filter technique may incorporate with the equation of the object motion pattern o that it can predict the occluded poition by learning the pattern of motion of the moving object. hi will definitely increae the efficiency of the enhanced Kalman filter to track and predict the poition of the moving object in occluion, not only in ame direction and velocity, but alo motion with pattern. REFERENCES [1] Fieguth, P. and erzopoulo, D. Color baed tracking of head and other mobile object at video frame rate. IEEE Conference on Computer Viion and Pattern Recognition (CVPR), pp , [2] Canny, J. A computational approach to edge detection. IEEE ran. Patt. Analy. Mach. Intell. 8(6), pp , [3] Moravec, H. Viual mapping by a robot rover. In Proceeding of the International Joint Conference on Artificial Intelligence (IJCAI) pp , [4] Harri,C. and Stephen, M. A combined corner and edge detector. In 4th Alvey Viion Conference, pp [5] Shi,J. and omai, C. Good feature to track. In IEEE Conference on Computer Viion and Pattern Recognition (CVPR), pp , [6] Lowe, D. Ditinctive image feature from cale-invariant key point. Int. J. Comput. Viion 60(2), pp , [7] Veenman, C., Reinder, M., and Backer, E. Reolving motion correpondence for denely moving point. IEEE ran. Patt. Analy. Mach. Intell. 23(1), pp , 2001.