Tracking and Recognition Presented by: Amir Rosenberger Instructor : Dr. Lior Wolf Support Vector Tracking, Shai Avidan, C...
What is Tracking? <ul><li>Problem of estimating the trajectory of an object in the image plane as it moves around a scene....
Challenges <ul><li>Problems: </li></ul><ul><ul><li>Noise in images </li></ul></ul><ul><ul><li>Complex object motion </li><...
Articles <ul><li>Support Vector Tracking (CVPR 2001) </li></ul><ul><li>Ensemble Tracking  (CVPR 2005) </li></ul><ul><li>Bo...
SVT <ul><li>Input </li></ul><ul><ul><li>Farme at time T+1 </li></ul></ul><ul><ul><li>Initial guess of the  </li></ul></ul>...
SVT Algorithm
SVT Algorithm Find a solution to
SVT Algorithm
Implementation <ul><ul><li>Iterations are made to refine result. </li></ul></ul><ul><ul><li>Use of Pyramid SVT </li></ul><...
Implementation <ul><li>Training: </li></ul><ul><ul><li>Reduce all images to 20x20 pixels image </li></ul></ul><ul><ul><li>...
<ul><li>Initial Guess Demonstration </li></ul>
<ul><li>Initial Guess Demonstration </li></ul>
<ul><li>Initial Guess Demonstration </li></ul>Initial: -8.3645 Final:  2.9272
Results Frame # SVM score
Results Frame # SVM score
Results Frame # SVM score
SVT <ul><li>Questions? </li></ul>
Ensemble Tracking <ul><li>Use of classifiers to distinguish the object from the background.  </li></ul><ul><li>Classifiers...
Ensemble Tracking <ul><li>Advantages </li></ul><ul><li>Breaks the time consuming training phase into a sequence of simple ...
General Algorithm <ul><li>Initialization (for frame  I 1 ): </li></ul><ul><li>Train several weak classifiers and add them ...
General Algorithm <ul><li>Label pixels inside rectangle  r j   as object and all those outside it as background </li></ul>...
Train: Feature Space Time  T
Test: Feature Space Time  T+1
Track: Time  T+1
Train Feature Space Time  T+1
Train Feature Space Time  T+1
Feature vector for every pixel <ul><li>Use: </li></ul><ul><li>8 bin local edge orientation histogram </li></ul><ul><li>Col...
Object and Background <ul><li>Pixels in a rectangle that bounds the object belongs to the object </li></ul><ul><li>Pixels ...
Confidence Measure <ul><li>weak classifier    confidence measure </li></ul><ul><li>[-∞, +∞]    [0, 1] </li></ul><ul><li>...
Mean Shift <ul><li>Mean shift procedure: </li></ul><ul><ul><li>1.Choose the radius r for the search window </li></ul></ul>...
Intuitive Description Distribution of identical billiard balls Region of interest Center of mass Mean Shift vector Objecti...
Intuitive Description Distribution of identical billiard balls Region of interest Center of mass Mean Shift vector Objecti...
Intuitive Description Distribution of identical billiard balls Region of interest Center of mass Mean Shift vector Objecti...
Intuitive Description Distribution of identical billiard balls Region of interest Center of mass Mean Shift vector Objecti...
Intuitive Description Distribution of identical billiard balls Region of interest Center of mass Mean Shift vector Objecti...
Intuitive Description Distribution of identical billiard balls Region of interest Center of mass Mean Shift vector Objecti...
Intuitive Description Distribution of identical billiard balls Region of interest Center of mass Objective  : Find the den...
Integration over time 1 3 2 4 5 6
Integration over time 1 3 2 4 5 6
Learning by AdaBoost <ul><li>For each weak learner, determine its error rate, given as:  </li></ul><ul><li>w i   is pixel ...
Learning by AdaBoost <ul><li>Weight of each weak learner is determined by error rate:  </li></ul><ul><ul><li>Classifiers w...
Learning by AdaBoost <ul><li>Pixel weights are increased if pixel is misclassified: </li></ul><ul><ul><li>Weight updated a...
Learning by AdaBoost <ul><li>After all weak learners are properly weighted, strong classifier defined as: </li></ul>
Outlier Rejection <ul><li>AdaBoost is sensitive to outliers </li></ul><ul><li>Threshold is used to change labels of “diffi...
Multi-resolution Tracking <ul><li>Scale the image to half and quarter </li></ul><ul><li>Use different ensembles independen...
Tracking results
Tracking results
Changing weights Frame 10 Frame 40 Frame 70 Color Histogram Weak classifiers
Comparing fixed Vs. dynamic ensemble
Handle moving camera
even grayscale sequences
Handling Occlusions
Ensemble tracking <ul><li>Questions? </li></ul>
Summary <ul><li>Two algorithms: </li></ul><ul><ul><li>Offline learning vs. online learning </li></ul></ul><ul><ul><li>Clas...
The End
Upcoming SlideShare
Loading in …5
×

Shai Avidan's Support vector tracking and ensemble tracking

4,113 views
3,918 views

Published on

CVPR 2001 and CVPR 2005 Shai Avidan's papers
presented by Amir Rosenberger

Published in: Technology, Business
1 Comment
4 Likes
Statistics
Notes
No Downloads
Views
Total views
4,113
On SlideShare
0
From Embeds
0
Number of Embeds
64
Actions
Shares
0
Downloads
0
Comments
1
Likes
4
Embeds 0
No embeds

No notes for slide
  • Shai Avidan's Support vector tracking and ensemble tracking

    1. 1. Tracking and Recognition Presented by: Amir Rosenberger Instructor : Dr. Lior Wolf Support Vector Tracking, Shai Avidan, CVPR 2001 Ensemble Tracking, Shai Avidan, CVPR 2005 Shai Avidan is credited for most of this presentation’s slides
    2. 2. What is Tracking? <ul><li>Problem of estimating the trajectory of an object in the image plane as it moves around a scene. </li></ul><ul><li>Tracking provides: </li></ul><ul><ul><li>Constants labels to objects </li></ul></ul><ul><ul><li>Object centric information </li></ul></ul>
    3. 3. Challenges <ul><li>Problems: </li></ul><ul><ul><li>Noise in images </li></ul></ul><ul><ul><li>Complex object motion </li></ul></ul><ul><ul><li>Camera Movements </li></ul></ul><ul><ul><li>Partial and full occlusions </li></ul></ul><ul><ul><li>Scene illuminations changes </li></ul></ul><ul><ul><li>Real time processing requirements </li></ul></ul>
    4. 4. Articles <ul><li>Support Vector Tracking (CVPR 2001) </li></ul><ul><li>Ensemble Tracking (CVPR 2005) </li></ul><ul><li>Both works were done by Shai Avidan </li></ul>
    5. 5. SVT <ul><li>Input </li></ul><ul><ul><li>Farme at time T+1 </li></ul></ul><ul><ul><li>Initial guess of the </li></ul></ul><ul><ul><li>object in time T </li></ul></ul><ul><li>Process </li></ul><ul><ul><li>Calculate SVM scores in the local surrounding and return a rectangle with max score </li></ul></ul>
    6. 6. SVT Algorithm
    7. 7. SVT Algorithm Find a solution to
    8. 8. SVT Algorithm
    9. 9. Implementation <ul><ul><li>Iterations are made to refine result. </li></ul></ul><ul><ul><li>Use of Pyramid SVT </li></ul></ul><ul><ul><li>Use homogeneous quadratic polynomial kernel </li></ul></ul><ul><ul><li>Scale of rectangle to SV size </li></ul></ul>
    10. 10. Implementation <ul><li>Training: </li></ul><ul><ul><li>Reduce all images to 20x20 pixels image </li></ul></ul><ul><ul><li>Classification </li></ul></ul><ul><ul><li> ~10,000 examples with 97% equal error rate </li></ul></ul><ul><ul><li>Test </li></ul></ul><ul><ul><li> ~10,000 examples with 92% equal error rate </li></ul></ul><ul><ul><li>Total of ~2000 Support Vectors, reduced to only 50 </li></ul></ul>
    11. 11. <ul><li>Initial Guess Demonstration </li></ul>
    12. 12. <ul><li>Initial Guess Demonstration </li></ul>
    13. 13. <ul><li>Initial Guess Demonstration </li></ul>Initial: -8.3645 Final: 2.9272
    14. 14. Results Frame # SVM score
    15. 15. Results Frame # SVM score
    16. 16. Results Frame # SVM score
    17. 17. SVT <ul><li>Questions? </li></ul>
    18. 18. Ensemble Tracking <ul><li>Use of classifiers to distinguish the object from the background. </li></ul><ul><li>Classifiers are used to create a confidence map </li></ul><ul><li>Training of new weak classifiers and adding them to the ensemble of weak classifiers </li></ul>
    19. 19. Ensemble Tracking <ul><li>Advantages </li></ul><ul><li>Breaks the time consuming training phase into a sequence of simple learning tasks that can be performed on-line </li></ul><ul><li>Can integrate off-line and on-line learning seamlessly. </li></ul><ul><li>Stability of tracker is improved in cases of partial occlusions or illumination changes. </li></ul>
    20. 20. General Algorithm <ul><li>Initialization (for frame I 1 ): </li></ul><ul><li>Train several weak classifiers and add them to the ensemble </li></ul><ul><li>For each new frame I j do: </li></ul><ul><li>Test all pixels in frame I j using the current strong classifier and create a confidence map L j </li></ul><ul><li>Run mean shift on the confidence map L j and report new object rectangle r j </li></ul>
    21. 21. General Algorithm <ul><li>Label pixels inside rectangle r j as object and all those outside it as background </li></ul><ul><li>Remove old weak classifiers </li></ul><ul><li>Train new weak classifiers on frame I j and add them to the ensemble </li></ul>
    22. 22. Train: Feature Space Time T
    23. 23. Test: Feature Space Time T+1
    24. 24. Track: Time T+1
    25. 25. Train Feature Space Time T+1
    26. 26. Train Feature Space Time T+1
    27. 27. Feature vector for every pixel <ul><li>Use: </li></ul><ul><li>8 bin local edge orientation histogram </li></ul><ul><li>Color (R,G,B) </li></ul><ul><li>This gives an 11D feature space to work in </li></ul>
    28. 28. Object and Background <ul><li>Pixels in a rectangle that bounds the object belongs to the object </li></ul><ul><li>Pixels outside that rectangle and inside a larger bounding rectangle belongs to the background </li></ul>
    29. 29. Confidence Measure <ul><li>weak classifier  confidence measure </li></ul><ul><li>[-∞, +∞]  [0, 1] </li></ul><ul><li>negative  0 </li></ul><ul><li>positive  (0, 1] </li></ul>
    30. 30. Mean Shift <ul><li>Mean shift procedure: </li></ul><ul><ul><li>1.Choose the radius r for the search window </li></ul></ul><ul><ul><li>2.Initialize the location of the window xk, k=1 </li></ul></ul><ul><ul><li>3.Compute the m.s.v. Mh,k(xk) </li></ul></ul><ul><ul><li>4.Translate the window by xk+1= Mh,k(xk)+xk, k=k+1 </li></ul></ul><ul><ul><li>5.Repeat step 3 and 4 until convergence </li></ul></ul>
    31. 31. Intuitive Description Distribution of identical billiard balls Region of interest Center of mass Mean Shift vector Objective : Find the densest region This sequence was taken from a presentation of Yaron Ukrainitz & Bernard Sarel on mean shift
    32. 32. Intuitive Description Distribution of identical billiard balls Region of interest Center of mass Mean Shift vector Objective : Find the densest region
    33. 33. Intuitive Description Distribution of identical billiard balls Region of interest Center of mass Mean Shift vector Objective : Find the densest region
    34. 34. Intuitive Description Distribution of identical billiard balls Region of interest Center of mass Mean Shift vector Objective : Find the densest region
    35. 35. Intuitive Description Distribution of identical billiard balls Region of interest Center of mass Mean Shift vector Objective : Find the densest region
    36. 36. Intuitive Description Distribution of identical billiard balls Region of interest Center of mass Mean Shift vector Objective : Find the densest region
    37. 37. Intuitive Description Distribution of identical billiard balls Region of interest Center of mass Objective : Find the densest region
    38. 38. Integration over time 1 3 2 4 5 6
    39. 39. Integration over time 1 3 2 4 5 6
    40. 40. Learning by AdaBoost <ul><li>For each weak learner, determine its error rate, given as: </li></ul><ul><li>w i is pixel weight, h t (x i ) is classification of pixel x i , y i is the proper label of pixel x i </li></ul>
    41. 41. Learning by AdaBoost <ul><li>Weight of each weak learner is determined by error rate: </li></ul><ul><ul><li>Classifiers with better performance have higher weights </li></ul></ul><ul><ul><li>Classifiers with error rates at %50 or higher are no better than chance, and are discarded </li></ul></ul>
    42. 42. Learning by AdaBoost <ul><li>Pixel weights are increased if pixel is misclassified: </li></ul><ul><ul><li>Weight updated as: </li></ul></ul>
    43. 43. Learning by AdaBoost <ul><li>After all weak learners are properly weighted, strong classifier defined as: </li></ul>
    44. 44. Outlier Rejection <ul><li>AdaBoost is sensitive to outliers </li></ul><ul><li>Threshold is used to change labels of “difficult” examples: </li></ul>
    45. 45. Multi-resolution Tracking <ul><li>Scale the image to half and quarter </li></ul><ul><li>Use different ensembles independently to create confidence maps </li></ul><ul><li>Average all the maps to one map on which the mean shift is performed </li></ul>
    46. 46. Tracking results
    47. 47. Tracking results
    48. 48. Changing weights Frame 10 Frame 40 Frame 70 Color Histogram Weak classifiers
    49. 49. Comparing fixed Vs. dynamic ensemble
    50. 50. Handle moving camera
    51. 51. even grayscale sequences
    52. 52. Handling Occlusions
    53. 53. Ensemble tracking <ul><li>Questions? </li></ul>
    54. 54. Summary <ul><li>Two algorithms: </li></ul><ul><ul><li>Offline learning vs. online learning </li></ul></ul><ul><ul><li>Classifiers of SV vs. ensemble of weak classifiers </li></ul></ul><ul><ul><li>Both use multi-resolution methods </li></ul></ul>
    55. 55. The End

    ×