Real-Time Tracking via On-line Boosting Helmut Grabner, Michael Grabner, Horst Bischof

Transcription

1 Real-Time Tracking via On-line Boosting, Michael Grabner, Horst Bischof Graz University of Technology Institute for Computer Graphics and Vision

2 Tracking Shrek M Grabner, H Grabner and H Bischof Real-time tracking with on-line feature selection CVPR 2006 Edinburgh, Sep 05, 2006

3 Tracking Requirements Adaptivity Appearance changes (eg out of plane rotations) Robustness Occlusions, cluttered background, illumination conditions Generality Any object Edinburgh, Sep 05, 2006

4 Outline Tracking as Classification Boosting for Feature selection From Off-line to On-line On-line Feature Selection Tracking Experimental Results Conclusion Edinburgh, Sep 05, 2006

5 Tracking as Classification Tracking as binary classification S Avidan Ensemble tracking CVPR 2005 JWang, et al Online selecting discriminative tracking features using particle filter CVPR 2005 object vs background Edinburgh, Sep 05, 2006

6 Tracking as Classification Tracking as binary classification problem S Avidan Ensemble tracking CVPR 2005 JWang, et al Online selecting discriminative tracking features using particle filter CVPR 2005 Object and background changes are robustly handled by on-line updating! object vs background Edinburgh, Sep 05, 2006

7 Boosting for Feature Selection Object Detector P Viola and M Jones Rapid object detection using a boosted cascade of simple features CVPR 2001 Fixed Training set General object detector Combination of simple image features using Boosting as Feature Selection Object Tracker On-line update Object vs Background On-Line Boosting for Feature Selection H Grabner and H Bischof On-line boosting and vision CVPR, 2006 Edinburgh, Sep 05, 2006

8 Off-line Boosting - set of labeled training samples - weight distribution over them Algorithm: - train a weak classifier using samples and weight dist - calculate error - calculate weight - update weight dist Y Freund and R Schapire A decision-theoretic generalization of on-line learning and an application to boosting Journal of Computer and System Sciences, 1997 Edinburgh, Sep 05, 2006

9 Off-line Boosting - set of labeled training samples - weight distribution over them Algorithm: - train a weak classifier using samples and weight dist - calculate error - calculate weight - update weight dist Edinburgh, Sep 05, 2006

10 Off-line Boosting - set of labeled training samples - weight distribution over them Algorithm: - train a weak classifier using samples and weight dist - calculate error - calculate weight - update weight dist Edinburgh, Sep 05, 2006

11 Off-line Boosting - set of labeled training samples - weight distribution over them Algorithm: - train a weak classifier using samples and weight dist - calculate error - calculate weight - update weight dist Edinburgh, Sep 05, 2006

12 Off-line Boosting - set of labeled training samples - weight distribution over them Algorithm: - train a weak classifier using samples and weight dist - calculate error - calculate weight - update weight dist Edinburgh, Sep 05, 2006

13 Off-line Boosting - set of labeled training samples - weight distribution over them Algorithm: - train a weak classifier using samples and weight dist - calculate error - calculate weight - update weight dist Result: Edinburgh, Sep 05, 2006

14 From Off-line to On-line Boosting off-line on-line - set of labeled training samples - weight distribution over them - train a weak classifier using samples and weight dist - calculate error - calculate weight - update weight dist Edinburgh, Sep 05, 2006

15 From Off-line to On-line Boosting off-line on-line - set of labeled training samples - weight distribution over them - train a weak classifier using samples and weight dist - calculate error - calculate weight - update weight dist Edinburgh, Sep 05, 2006

16 From Off-line to On-line Boosting off-line only one training example to update the classifier on-line - set of labeled training samples - ONE labeled training sample - weight distribution over them - strong classifier to update - train a weak classifier using samples and weight dist - calculate error - calculate weight - update weight dist Edinburgh, Sep 05, 2006

17 From Off-line to On-line Boosting off-line update importance for the current sample on-line - set of labeled training samples - ONE labeled training sample - weight distribution over them - strong classifier to update - train a weak classifier using samples and weight dist - initial importance - calculate error - calculate weight - update weight dist - update importance weight Edinburgh, Sep 05, 2006

18 From Off-line to On-line Boosting off-line online update the weak classifier on-line - set of labeled training samples - ONE labeled training sample - weight distribution over them - strong classifier to update - train a weak classifier using samples and weight dist - initial importance - update the weak classifier using samples and importance - calculate error - calculate weight - update weight dist - update importance weight Edinburgh, Sep 05, 2006

19 From Off-line to On-line Boosting off-line update errors and weights on-line - set of labeled training samples - ONE labeled training sample - weight distribution over them - strong classifier to update - train a weak classifier using samples and weight dist - initial importance - update the weak classifier using samples and importance - calculate error - calculate weight - update weight dist - update error estimation - update weight - update importance weight Edinburgh, Sep 05, 2006

20 From Off-line to On-line Boosting off-line - set of labeled training samples on-line - ONE labeled training sample - weight distribution over them - strong classifier to update - train a weak classifier using samples and weight dist - initial importance - update the weak classifier using samples and importance - calculate error - calculate weight - update weight dist - update error estimation - update weight - update importance weight Edinburgh, Sep 05, 2006

21 On-line Boosting - ONE labeled training sample - strong classifier to update Algorithm: - initial importance - update the weak classifier using sample and importance - update error estimation - update weight - update importance weight Edinburgh, Sep 05, 2006

22 On-line Boosting - ONE labeled training sample - strong classifier to update Algorithm: - initial importance - update the weak classifier using sample and importance - update error estimation - update weight - update importance weight Edinburgh, Sep 05, 2006

23 On-line Boosting - ONE labeled training sample - strong classifier to update Algorithm: - initial importance - update the weak classifier using sample and importance - update error estimation - update weight - update importance weight Edinburgh, Sep 05, 2006

24 On-line Boosting - ONE labeled training sample - strong classifier to update Algorithm: - initial importance - update the weak classifier using sample and importance - update error estimation - update weight - update importance weight Edinburgh, Sep 05, 2006

25 On-line Boosting - ONE labeled training sample - strong classifier to update Algorithm: - initial importance - update the weak classifier using sample and importance - update error estimation - update weight - update importance weight Edinburgh, Sep 05, 2006

26 On-line Boosting - ONE labeled training sample - strong classifier to update Algorithm: - initial importance - update the weak classifier using sample and importance - update error estimation - update weight - update importance weight Edinburgh, Sep 05, 2006

27 On-line Boosting - ONE labeled training sample - strong classifier to update Algorithm: - initial importance Converges to the off-line results - update the weak classifier using sample and importance - update error estimation N Oza and S Russell Online bagging and boosting Artificial Intelligence and Statistics, update weight - update importance weight Result: Edinburgh, Sep 05, 2006

28 On-line Boosting for Feature Selection 1/3 Each feature corresponds to a weak classifier Features Haar-like wavelets Orientation histograms Locally binary patterns (LBP) Fast computation using efficient data structures integral images integral histograms F Porikli Integral histogram: A fast way to extract histograms in cartesian spaces CVPR 2005 Edinburgh, Sep 05, 2006

29 On-line Boosting for Feature Selection 2/3 Introducing Selector selects one feature from its local feature pool On-line boosting is performed on the Selectors and not on the weak classifiers directly H Grabner and H Bischof On-line boosting and vision CVPR, 2006 Edinburgh, Sep 05, 2006

30 On-line Boosting for Feature Selection 3/3 one traning sample Updating the weak h classifier is very 1,1 time consuming! h 1,2 hselector 1 hselector 2 hselector N h 2,1 h 2,2 h N,1 h N,2 inital importance importance importance h 2,m h N,m Use a shared feature pool h 1,M h 2,M h N,M update update update current strong classifier hstrong repeat for each trainingsample Edinburgh, Sep 05, 2006

31 Direct Feature Selection one traning sample h i h 1 h i h k h m h M h l gloabal weak classifer pool hselector 1 hselector 2 hselectorn inital importance errors select best weak classifier importance errors select best weak classifier importance errors select best weak classifier update weight update weight update weight repeat for each trainingsample current strong classifier hstrong Edinburgh, Sep 05, 2006

32 Direct Feature Selection one traning sample h i h 1 h i h k h m h M h l gloabal weak classifer pool hselector 1 hselector 2 hselectorn inital importance errors select best weak classifier importance errors select best weak classifier importance errors select best weak classifier update weight update weight update weight repeat for each trainingsample current strong classifier hstrong Edinburgh, Sep 05, 2006

33 Direct Feature Selection one traning sample h i h 1 h i h k h m h M h l gloabal weak classifer pool hselector 1 hselector 2 hselectorn inital importance errors select best weak classifier importance errors select best weak classifier importance errors select best weak classifier update weight update weight update weight repeat for each trainingsample current strong classifier hstrong Edinburgh, Sep 05, 2006

34 Direct Feature Selection one traning sample h i h 1 h i h k h m h M h l gloabal weak classifer pool hselector 1 hselector 2 hselectorn inital importance errors select best weak classifier importance errors select best weak classifier importance errors select best weak classifier update weight update weight update weight repeat for each trainingsample current strong classifier hstrong Edinburgh, Sep 05, 2006

35 Tracking 1/2 from time t to t+1 evaluate classifier on sub-patches actual object position search Region update classifier (tracker) analyze map and set new object position create confidence map Edinburgh, Sep 05, 2006

36 Tracking 2/2 Confidence Map Tracking Max Confidence Value Edinburgh, Sep 05, 2006

37 On-line Feature Exchange Edinburgh, Sep 05, 2006

38 Public Sequences J Lim, D Ross, R Lin, and M Yang Incremental learning for visual tracking NIPS 2005 A D Jepson, D J Fleet, and TF El-Maraghi Robust online appearance models for visual tracking CVPR 2001 Edinburgh, Sep 05, 2006

39 Tracking the Invisible Edinburgh, Sep 05, 2006

40 Conclusion Tracking as Classification Real-Time Continuously updating a classifier which discriminates the object from the background Adaptivity Robustness Generality Efficient data structures for all basic image features types Shared Feature Pool Edinburgh, Sep 05, 2006

41 Thank you for your attention Questions? Combination: Detection, Tracking and Recognition Edinburgh, Sep 05, 2006