Mean Shift Tracking

   CS4243 Computer Vision and Pattern Recognition

                     Leow Wee Kheng

             ...
Mean Shift


Mean Shift

Mean Shift [Che98, FH75, Sil86]
    An algorithm that iteratively shifts a data point to the aver...
Mean Shift


Consider a set S of n data points xi in d-D Euclidean space X.
Let K(x) denote a kernel function that indicat...
Mean Shift


The sequence x, m(x), m(m(x)), . . . is called the trajectory of x.
If sample means are computed at multiple ...
Mean Shift    Kernel


Kernel

      Typically, kernel K is a function of x 2 :

                                K(x) = k(...
Mean Shift   Kernel


Examples of kernels [Che98]:
    Flat kernel:
                                           1 if x ≤ 1
...
Mean Shift   Density Estimation


Density Estimation

   Kernel density estimation (Parzen window technique) is a popular
...
Mean Shift       Density Estimation


The mean square error is minimized by the Epanechnikov kernel:
                 
  ...
Mean Shift   Density Estimation


    Define another kernel G(x) = g( x 2 ) such that

                                    ...
Mean Shift         Density Estimation


Proof:
          ˜
    Define f with kernel G
                                     ...
Mean Shift     Density Estimation


Estimate of density gradient is the gradient of density estimate
        ˜ fK (x) ≡   ...
Mean Shift Tracking


Mean Shift Tracking

Basic Ideas [CRM00]:
    Model object using color probability density.
    Trac...
Mean Shift Tracking     Object Model


Object Model

  Let xi , i = 1, . . . , n, denote pixel locations of model centered...
Mean Shift Tracking   Object Model


δ is the Kronecker delta function
                                       1 if a = 0
 ...
Mean Shift Tracking     Target Candidate


Target Candidate

   Let yi , i = 1, . . . , nh , denote pixel locations of tar...
Mean Shift Tracking   Color Density Matching


Color Density Matching

   Use Bhattacharyya coefficient ρ
                  ...
Mean Shift Tracking      Tracking Algorithm


Tracking Algorithm

Given {qu } of model and location y of target in previou...
Mean Shift Tracking   Tracking Algorithm


Step 3: In practice, a window of pixels yi is considered.
Size of window is rel...
Mean Shift Tracking   Tracking Algorithm


Example 1: Track football player no. 78 [CRM00].




       (CS4243)           ...
Mean Shift Tracking   Tracking Algorithm


Example 2: Track a passenger in train station [CRM00].




       (CS4243)     ...
Reference


Reference I

  Y. Cheng.
  Mean shift, mode seeking, and clustering.
  IEEE Trans. on Pattern Analysis and Mac...
Reference


Reference II

   R. O. Duda and P. E. Hart.
   Pattern Classification and Scene Analysis.
   Wiley, 1973.
   K....
Upcoming SlideShare
Loading in...5
×

Meanshift Tracking Presentation

5,430
-1

Published on

Published in: Technology, Business
1 Comment
1 Like
Statistics
Notes
  • pretty well presentation. I have found what I'm seeking for! thx
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
5,430
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
202
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

Meanshift Tracking Presentation

  1. 1. Mean Shift Tracking CS4243 Computer Vision and Pattern Recognition Leow Wee Kheng Department of Computer Science School of Computing National University of Singapore (CS4243) Mean Shift Tracking 1 / 1
  2. 2. Mean Shift Mean Shift Mean Shift [Che98, FH75, Sil86] An algorithm that iteratively shifts a data point to the average of data points in its neighborhood. Similar to clustering. Useful for clustering, mode seeking, probability density estimation, tracking, etc. (CS4243) Mean Shift Tracking 2 / 1
  3. 3. Mean Shift Consider a set S of n data points xi in d-D Euclidean space X. Let K(x) denote a kernel function that indicates how much x contributes to the estimation of the mean. Then, the sample mean m at x with kernel K is given by n K(x − xi ) xi i=1 m(x) = n (1) K(x − xi ) i=1 The difference m(x) − x is called mean shift. Mean shift algorithm: iteratively move date point to its mean. In each iteration, x ← m(x). The algorithm stops when m(x) = x. (CS4243) Mean Shift Tracking 3 / 1
  4. 4. Mean Shift The sequence x, m(x), m(m(x)), . . . is called the trajectory of x. If sample means are computed at multiple points, then at each iteration, update is done simultaneously to all these points. (CS4243) Mean Shift Tracking 4 / 1
  5. 5. Mean Shift Kernel Kernel Typically, kernel K is a function of x 2 : K(x) = k( x 2 ) (2) k is called the profile of K. Properties of Profile: 1 k is nonnegative. 2 k is nonincreasing: k(x) ≥ k(y) if x < y. 3 k is piecewise continuous and ∞ k(x)dx < ∞ (3) 0 (CS4243) Mean Shift Tracking 5 / 1
  6. 6. Mean Shift Kernel Examples of kernels [Che98]: Flat kernel: 1 if x ≤ 1 K(x) = (4) 0 otherwise Gaussian kernel: K(x) = exp(− x 2 ) (5) (a) Flat kernel (b) Gaussian kernel (CS4243) Mean Shift Tracking 6 / 1
  7. 7. Mean Shift Density Estimation Density Estimation Kernel density estimation (Parzen window technique) is a popular method for estimating probability density [CRM00, CRM02, DH73]. For a set of n data points xi in d-D space, the kernel density estimate with kernel K(x) (profile k(x)) and radius h is n ˜ 1 x − xi fK (x) = K nhd h i=1 (6) n 2 1 x − xi = k nhd h i=1 The quality of kernel density estimator is measured by the mean square error between the actual density and the estimate. (CS4243) Mean Shift Tracking 7 / 1
  8. 8. Mean Shift Density Estimation The mean square error is minimized by the Epanechnikov kernel:   1 (d + 2)(1 − x 2 ) if x ≤ 1 KE (x) = 2Cd (7)  0 otherwise where Cd is the volume of the unit d-D sphere, with profile   1 (d + 2)(1 − x) if 0 ≤ x ≤ 1 kE (x) = 2Cd (8)  0 if x > 1 A more commonly used kernel is Gaussian 1 1 2 K(x) = exp − x (9) (2π)d 2 with profile 1 1 k(x) = exp − x (10) (2π)d 2 (CS4243) Mean Shift Tracking 8 / 1
  9. 9. Mean Shift Density Estimation Define another kernel G(x) = g( x 2 ) such that dk(x) g(x) = −k (x) = − (11) dx Important Result Mean shift with kernel G moves x along the direction of the gradient of ˜ density estimate f with kernel K. Define density estimate with kernel K. But, perform mean shift with kernel G. Then, mean shift performs gradient ascent on density estimate. (CS4243) Mean Shift Tracking 9 / 1
  10. 10. Mean Shift Density Estimation Proof: ˜ Define f with kernel G n 2 ˜ C x − xi fG (x) ≡ g (12) nhd h i=1 where C is a normalization constant. Mean shift M with kernel G is n 2 x − xi g xi h i=1 MG (x) ≡ n 2 −x (13) x − xi g h i=1 (CS4243) Mean Shift Tracking 10 / 1
  11. 11. Mean Shift Density Estimation Estimate of density gradient is the gradient of density estimate ˜ fK (x) ≡ ˜ fK (x) n 2 2 x − xi = (x − xi ) k nhd+2 h i=1 n 2 (14) 2 x − xi = (xi − x) g nhd+2 h i=1 2 ˜ = fG (x)MG (x) Ch2 Then, Ch2 ˜ fK (x) MG (x) = (15) ˜ 2 fG (x) Thus, MG is an estimate of the normalized gradient of fK . So, can use mean shift (Eq. 13) to obtain estimate of fK . (CS4243) Mean Shift Tracking 11 / 1
  12. 12. Mean Shift Tracking Mean Shift Tracking Basic Ideas [CRM00]: Model object using color probability density. Track target object in video by matching color density. Use mean shift to estimate color density and target location. (CS4243) Mean Shift Tracking 12 / 1
  13. 13. Mean Shift Tracking Object Model Object Model Let xi , i = 1, . . . , n, denote pixel locations of model centered at 0. Represent color distribution by discrete m-bin color histogram. Let b(xi ) denote the color bin of the color at xi . Assume size of model is normalized; so, kernel radius h = 1. Then, the probability q of color u in the model is n qu = C k( xi 2 ) δ(b(xi ) − u) (16) i=1 C is the normalization constant n −1 2 C= k( xi ) (17) i=1 Kernel profile k weights contribution by distance to centroid. (CS4243) Mean Shift Tracking 13 / 1
  14. 14. Mean Shift Tracking Object Model δ is the Kronecker delta function 1 if a = 0 δ(a) = (18) 0 otherwise That is, contribute k( xi 2 ) to qu if b(xi ) = u. (CS4243) Mean Shift Tracking 14 / 1
  15. 15. Mean Shift Tracking Target Candidate Target Candidate Let yi , i = 1, . . . , nh , denote pixel locations of target centered at y. Then, the probability p of color u in the target is nh 2 y − yi pu (y) = Ch k δ(b(yi ) − u) (19) h i=1 Ch is the normalization constant nh 2 −1 y − yi Ch = k (20) h i=1 (CS4243) Mean Shift Tracking 15 / 1
  16. 16. Mean Shift Tracking Color Density Matching Color Density Matching Use Bhattacharyya coefficient ρ m ρ(p(y), q) = pu (y) qu (21) u=1 √ √ √ √ ρ is the cosine of vectors ( p1 , . . . , pm ) and ( q1 , . . . , qm ) . Large ρ means good color match. For each image frame, find y that maximizes ρ. This y is the location of the target. (CS4243) Mean Shift Tracking 16 / 1
  17. 17. Mean Shift Tracking Tracking Algorithm Tracking Algorithm Given {qu } of model and location y of target in previous frame: 1 Initialize location of target in current frame as y. 2 Compute {pu (y)} and ρ(p(y), q). 3 Apply mean shift: Compute new location z as nh 2 y − yi g yi h i=1 z= nh 2 (22) y − yi g h i=1 4 Compute {pu (z)} and ρ(p(z), q). 5 While ρ(p(z), q) < ρ(p(y), q), do z ← 1 (y + z). 2 6 If z − y is small enough, stop. Else, set y ← z and goto (1). (CS4243) Mean Shift Tracking 17 / 1
  18. 18. Mean Shift Tracking Tracking Algorithm Step 3: In practice, a window of pixels yi is considered. Size of window is related to h. Step 5 is used to validate the target’s new location. Can stop Step 5 if y and z round off to the same pixel. Tests show that Step 5 is needed only 0.1% of the time. Step 6: can stop algorithm if y and z round off to the same pixel. To track object that changes size, varies radius h (see [CRM00] for details). (CS4243) Mean Shift Tracking 18 / 1
  19. 19. Mean Shift Tracking Tracking Algorithm Example 1: Track football player no. 78 [CRM00]. (CS4243) Mean Shift Tracking 19 / 1
  20. 20. Mean Shift Tracking Tracking Algorithm Example 2: Track a passenger in train station [CRM00]. (CS4243) Mean Shift Tracking 20 / 1
  21. 21. Reference Reference I Y. Cheng. Mean shift, mode seeking, and clustering. IEEE Trans. on Pattern Analysis and Machine Intelligence, 17(8):790–799, 1998. D. Comaniciu, V. Ramesh, and P. Meer. Real-time tracking of non-rigid objects using mean shift. In IEEE Proc. on Computer Vision and Pattern Recognition, pages 673–678, 2000. D. Comaniciu, V. Ramesh, and P. Meer. Mean shift: A robust approach towards feature space analysis. IEEE Trans. on Pattern Analysis and Machine Intelligence, 24(5):603–619, 2002. (CS4243) Mean Shift Tracking 21 / 1
  22. 22. Reference Reference II R. O. Duda and P. E. Hart. Pattern Classification and Scene Analysis. Wiley, 1973. K. Fukunaga and L. D. Hostetler. The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Trans. on Information Theory, 21:32–40, 1975. B. W. Silverman. Density Estimation for Statistics and Data Analysis. Chapman and Hall, 1986. (CS4243) Mean Shift Tracking 22 / 1
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×