Some discussion about real time object tracking and detection methods.

1. REAL-TIME OBJECT DETECTION
AND TRACKING
By:
Vanya V. Valindria
Hammad Naeem
Rui Hua

2. Outline
• Introduction
• Hardware in RT Object Detetion & Tracking
• Methods
Traditional Methods: Modern Methods:
Absolute Differences
Census Method KLT
Feature Based Method Meanshift
• Result and Conclusion

3. Introduction
Definition:
Object detection
 detect a particular
object in an image
Object tracking
 to track an object
(or multiple objects)
over a sequence of
images

4. Application: Traffic Information
http://www.youtube.com/watch?v=vA35sXbn7zs

5. Application: Surveillance
http://www.youtube.com/watch?v=o25fClk9cdg

6. Application: Mobile Robot
http://www.youtube.com/watch?v=Q4zycRGJFFs

7. Problems??
• Temporal variation/dynamic environment
• Abrupt object or camera motion
• Multi-camera? Multi-objects?
• Computational expensive

8. Hardware in Real-time Tracking
• MEMORY
Important  Tracking system encountering limited memory
problems.
• FRAME RATE
~30 FPS
• PROCESSORS - DSP
• Allow saturated arithmetic operation
• Powerful operation ability
• Can do several memory accesses in a single instruction

10. Object Detection and Tracking
• In a video sequence an object is said to be in
motion, if it is changing its location with
respect to its background
• The motion tracking is actually the process of
keeping tracks of that moving object in video
sequence i.e. position of moving object at
certain time etc.

11. Flow Chart
Idle
Image
acquisition
Object
Detection
Image
acquisition
Object
tracking
Object
No
Lost?
Y es

12. Method 1: Absolute Differences
= Image subtraction
D(t)=I(ti) – I(tj)
Gives an image frame with changed and
unchanged regions
Ideal Case for no motion: I(ti) = I(tj), D(t)=0

13. Moving
objects
are
detected

14. Results:
Frame1 Frame10
Difference of Two Frames

15. Absolute Difference
Methods for Motion Detection
 Frame Differencing
 Background Subtraction
Draw Backs:
involves a lot of computations
 Not feasible for DSP implementation

16. Method 2: Census Transforms
124 74 32 1 1 0
124 64 18 If (Center pixel < Neighbor
1 x 0
pixel)
Neighbor pixel = 1
157 116 84 1 1 1
Signature Vector Generation
1 1 0
1 x 0
Signature Vector
11011101 1 1 1

17. Image
128 26 125 243 87 Signature Vectors
96 76 43 236 125
10110101
00101011
128 129 235 229 209 Signatur vector generation for
.
all pixels
228 251 229 221 234
.
.
227 221 35 58 98 10111010
List 10110101
Generation List
00101011 population
.
.
.
Generated List 10111010
Signature vector
matching

18. Census Transform:
Advantages:
 Compare only two values 0 or 1.
 Similar Illumination Variation for pixel and
neighbouring pixels
Draw Backs:
 As we only deal with only 0`s and 1`s, this method is
sensitive to noise.
 Calculate, store and match process  computationally
Expensive

19. Method 3: Morphology Based
Object Tracking
Background Frame Object
estimation differencing Registration

20. Morphology Based Object Tracking
• Image Differencing
Background • Thresholding
Estimation
• Contours are registered
Object • Width, height and histogram are recorded for each contour
Registration
• Each object represented by a feature vector (the length,
Feature width, area and histogram of the object)
Vector

21. Segmented object
Tracked object

22. Morphology Based Techniques
Advantages:
Can Track Multiple objects  Objects are registered
based on their anatomy
 Helpful for Object Merging
Draw Backs:
Object registration  complex and slow process
 For multiple object registration per frame  more
complex

23. Method 4: Lucas-Kanade Technique
• Visual motion pattern of objects and surface in a
scene  by Optical Flow
Frame 1 Frame 2

24. Method 5: Mean shift
• An algorithm that iteratively shifts a data point to the
average of data points in its neighborhood
Choose a search window Compute the MEAN
size in the initial location location in the
search window
Repeat until Center the search
convergence window
at the mean

25. Intuitive Description
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Distribution of identical balls

26. Intuitive Description Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Distribution of identical balls

27. Intuitive Description
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Distribution of identical balls

28. Intuitive Description
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Distribution of identical balls

29. Intuitive Description
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Distribution of identical balls

30. Intuitive Description
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Distribution of identical balls

31. Intuitive Description
Region of
interest
Center of
mass
Objective : Find the densest region
Distribution of identical balls

32. Process

33. CAMSHIFT
--Continously Adaptive Meanshift
Modified to adapt dynamically to the colour
probability distributions
More real time
For each frame-> MEAN-SHIFT is
applied with several iteration
Store the location of the mean and
calculate new window size for next
frame

34. New development
• Combine with different features. SIFT features,
colour feature & texture information
• Camshift algorithm combined with the Kalman
filter.

35. Result
Arithmetic and Time taken
Algorithm Logic by
operations Algorithm
Absolute
4230100 16
Differencing
Census
2416000 5. 4
Transform
Morphological
352210 14.2
Tracking
Kanade Lucas 500825 0.486

36. Comparison
Computationally
Easy to implement
expensive
Absolute Differences Allows continuous Slow and low
tracking accuracy
Computationally
expensive
Census Transform Immune to noise and
Illumination changes Complex if 
Multiple objects
per frame
Can track multiple Slow
Feature Based
objects well
Large Memory
consumption

37. Comparison
High accuracy
KLT
Less execution time Large memory
Robust to noise and
dynamic scene
Ineffective if
Computationally less there is heavy
MeanShift & CAMShift expensive occlusion

38. Conclusion
• KLT algorithm has the best performance with
higher accuracy and less computation time
• It requires combination of methods to achieve
the appropriate object detection and tracking
according to the proposed scenario

39. References
• S. Shah, T. Khattak, M. Farooq, Y. Khawaja, A. Bais, A. Anees, and M. Khan, “Real Time Object
Tracking in a Video Sequence Using a Fixed Point DSP,” Advances in Visual Computing, pp. 879–
888.
• K. Huang, L. Wang, T. Tan, and S. Maybank, “A real-time object detecting and tracking system
for outdoor night surveillance,” Pattern Recognition, vol. 41, no. 1, pp. 432–444, 2008.
• J. Li, F. Li, and M. Zhang, “A Real-time Detecting and Tracking Method for Moving Objects Based
on Color Video,” in 2009 Sixth International Conference on Computer Graphics, Imaging and
Visualization. IEEE, 2009, pp. 317–322.
• W. Junqiu and Y. Yagi, “Integrating color and shapetexture features for adaptive real-time
object tracking,” IEEE Trans on Image Processing, vol. 17, no. 2, pp. 235–240, 2008.
• Q. Wang and Z. Gao, “Study on a Real-Time Image Object Tracking System,” in Computer
Science and Computational Technology, 2008. ISCSCT’08. International Symposium on, vol. 2,
2008.
• Y. Meng, “Agent-based reconfigurable architecture for real-time object tracking,” Journal of
Real-Time Image Processing, vol. 4, no. 4, pp. 339–351, 2009.
• [Y. Yao, C. Chen, A. Koschan, and M. Abidi, “Adaptive online camera coordination for multi-
camera multi-target surveillance,” Computer Vision and Image Understanding, 2010.