This document summarizes object tracking methods, including representations of objects, features for tracking, detection approaches, tracking algorithms, and future directions. It discusses representing objects as points, patches, or contours, using features like color, edges, texture, and optical flow for detection and tracking. Detection can be done through point detection, background subtraction, segmentation, and supervised learning. Tracking algorithms include point tracking, kernel tracking, and silhouette tracking. The document outlines challenges like occlusion, camera motion, and non-rigid objects that remain for future work in object tracking.

1. object tracking:a surveyNhat ‘Rich’ NguyenVision SeminarFebruary 2010Based on a paper by Yilmaz et al

2. Definition2Tracking is the problem of estimatingthe trajectory of an object in the image plane as it moves around a scene.

3. Applications3Motion RecognitionAutomated SurveillanceVideo IndexingHuman Computer InteractionTraffic MonitoringVehicleNavigation

4. ProblemsProjectionNoisesComplex shapeComplex motionNon-rigid OcclusionsLightingReal-time4

5. QuestionsWhich object representation is suitable?Which image features should be used?How should motion, appearance of the object be modeled?5Help you to design an object tracking system

6. OverviewObject RepresentationsFeatures SelectionObject DetectionObject TrackingFuture Direction6

7. 1. Object Representation7[How to represent an object for tracking]

8. 8Shape - PointsCentroidMultiple PointsControl Points

9. 9Shape - PatchesRectangular PatchEllipticalPatchMultiple Patches

10. 10Shape - ContourCompleteContourSkeletal ModelSilhouette

11. 11Appearance – Prob. DensitiesGaussianHistogramMixture of Gaussians

12. 12Appearance – ModelsGeometric TemplateActive ContourMulti-view Appearance

13. 2. Feature Selection13[Which feature can be easily distinguished?]

14. 14ColorHSVRGBLAB

15. 15EdgesCanny Edge Detector

16. 16Optical FlowDense field of displacement vectors which defines the translation of each pixel

17. 17TextureGray-level Co-occurence Matrix

18. 18Texture – Law’s measures1-D: kernel for Level, Edge, Spot, Wave, and Ripple2-D: convoluting a vertical and a horizontal 1-D kernel

19. 3. Object Detection19[To track, we first detect.]

20. 20ApproachesPoint DetectorHarris SIFTBackground SubtractionSegmentationMean shiftGraph cutsActive ContoursSupervised LearningAdaptive BoostingSupport Vector Machines

21. 21Point DetectorsHarrisSIFT

22. 22Background Subtraction

23. 23Segmentation

24. 24Segmentation - Mean shift

25. 25Segmentation - Mean shift

26. 26Segmentation – Graph-cuts

27. 27Segmentation – Active Contour

28. 28Supervised LearningLearningExamplesFeaturesSupervised LearnersInputClassification

29. 29Adaptive Boosting

30. 30Support Vector Machine

31. 4. Object Tracking31[State-of-the-art methods.]

32. 32ApproachesPoint Tracking[Multi-point Correspondence] Kernel Tracking[Parametric Transformation] Silhouette Tracking[Contour Evolution]

33. 33Taxonomy

34. 34DeterministicAll possible AssociationsUniqueAssociationsMulti-frameCorrespondenceOptimal Assignment Methods:Hungarian vs. Greedy

35. 35Motion ConstraintsProximitySmall changein velocityMaximumVelocityCommonMotionRigidity

36. 36ExamplesRotating dishFlying birds

37. 37State Estimation

38. Estimate the state of a linear system.The state is Gaussian distributed.Filters38KalmanThe state is NOT Gaussian distributed.ParticleInstead of nearest neighbor, offer a probabilistic approach for data associationNo entering or exiting objectsJoint ProbabilityData AssociationMultiple HypothesisExhaustively enumerate all possible associations.

39. 39Evaluation

40. 40Template MatchingBrute forceSimilarity measure: cross correlation- specifies candidate template position- object template in previous frame

41. 41Mean Shift Tracker

42. 42KLT Feature TrackerCompute the translation of a rectangular region centered on an interest point.Evaluate the quality by computing the affine transformation between corresponding patches.

43. 43Eigen TrackerSubspace-based approach for multi-view appearance.Uses eigenspace for similarity instead of SSD, or correlation.Allows distortion in the template.

44. 44SVM TrackerPositive samples consist of images of the object to be tracked.Negative samples consist of images of background object.Maximizes the SVM classification score over image region to estimate the object position.Knowledge about background object is explicitly incorporated in the tracker.

45. 45Evaluation

46. 46Shape MatchingSimilar to Template MatchingUse Hausdorff distance measure to identify most mismatch edges.Emphasize parts of model that are not drastically affected by object motion.Examples of a person walking : head and torso vs. arms and legs.

47. 47State Space ModelState is term of shape and motion parameters of the contourControl points of the contour moves on the spring stiffness parametersMeasurements consist of the image edges computed in the normal direction of the contour

48. 48Gradient DescentDirect minimization algorithm.

49. To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient of the function at the current point.rms and legs.49Contour Evolution

50. 50Evaluation

51. 5. Future Direction51[What’s left for us?]

52. Depth informationOcclusion ResolutionMoving camerasNon-overlapping viewMultiple Camera Tracking52

53. Broadcast news or home videos.Noisy, compressed, unstructured, multiple views.Severe occlusion, object partially visible.Employ audio in addition to video.Unconstrained Videos53

54. Ability to learn object model online.Unsupervised learning of object models for multiple non-rigid moving object from a single camera.Efficient Online Estimation54

55. Require detection at some point.State-of-the-art tracking methods.Point correspondenceGeometric modelsContour evolutionDependency on context of use.Give valuable insight and encourage new research.Concluding Remarks55