Real-Time Logo Detection and Tracking


Published on

A computationally efficient method to detect and track logo’s in video

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Real-Time Logo Detection and Tracking

  1. 1. SPIE Conference on Real-Time Image and Video Processing April 16, 2010 - Brussels M. George a , N. Kehtarnavaz a , M. Rahman a , M. Carlsohn b a Signal and Image Processing Lab, University of Texas at Dallas b Engineering and Consultancy for Computer Vision and Image Communication, Bremen, Germany This work has been partially supported by the Wireless Terminal Business Unit of Texas Instruments.
  2. 2. <ul><li>Motivation for this work </li></ul><ul><li>Existing approaches </li></ul><ul><li>Logo detection using SIFT </li></ul><ul><li>Real-time logo detection/tracking using online color calibration </li></ul><ul><li>Detection results/videoclips </li></ul>
  3. 3. <ul><li>User demands for value added applications on smart phones are increasing </li></ul><ul><li>Logo detection can be used to provide consumers with offers linked to logos </li></ul><ul><li>Logo detection can also be used together with GPS location services </li></ul>
  4. 4. <ul><ul><li>Challenges: </li></ul></ul><ul><ul><li>Should work for any size logo (smart phone camera seeing logos at different distances), for any logo orientation (holding smart phone camera at any angle), under any lighting condition </li></ul></ul><ul><ul><li>Detection methods that can accommodate different sizes and orientations: </li></ul></ul><ul><ul><li>Moment Invariance (specific location is needed, otherwise background objects would make it fail) </li></ul></ul><ul><ul><li>Viola and Jones (training very time consuming for various orientations) </li></ul></ul><ul><ul><li>Scale Invariant Feature Transform (SIFT) – most promising, widely used for object detection applications, but is slow </li></ul></ul><ul><ul><li>Our contribution in this paper has been on the real-time aspect </li></ul></ul><ul><ul><li>Introducing a hybrid approach by combining SIFT for initial detection and a computationally efficient online color calibration and moment invariants for subsequent detection </li></ul></ul>
  5. 5. <ul><li>Robust object detection technique introduced by David Lowe (1999) </li></ul><ul><li>Able to detect objects at different scales making it scale invariant </li></ul><ul><li>Descriptors using orientation histograms provide rotation invariance </li></ul>
  6. 6. <ul><li>Pyramid of images generated by Gaussian smoothing and subsampling </li></ul><ul><li>Difference of Gaussian (DoG) calculated </li></ul><ul><li>Maxima and minima points in DoG images are used to denote keypoints </li></ul>Figure showing octaves, levels within octaves and Difference of Gaussian (DOG) for scale space extrema* *Taken from “Distinctive Image Features from Scale-Invariant Keypoints” by David Lowe (2005)
  7. 7. SIFT keypoints marked <ul><li>Gradient magnitude and orientation calculated </li></ul><ul><li>360° orientation histogram uses gradient orientations of all neighboring pixels around keypoints </li></ul><ul><li>128-dimensional SIFT descriptor vector provides location, scale and orientation information </li></ul><ul><li>Matching of descriptors done through Best Bin First Search (k-d tree search variant). </li></ul>keypoints with gradient and orientation information
  8. 8. <ul><li>SIFT is computationally intensive so here it is just used for initial detection </li></ul><ul><li>Subsequent detection or tracking is done using color and moment invariants </li></ul><ul><li>K-means clustering is applied to the SIFT logo detected region in order to extract the logo color under the light source the image frame is taken </li></ul><ul><li>Online calibrated color is then used to detect/track the logo in subsequent frames </li></ul><ul><li>Moment invariants applied to all regions having similar colors to increase robustness of detection </li></ul>
  9. 9. <ul><li>Color is a very effective feature but has the problem of being dependent on the light source (color temperature) under which the image is taken. By using online color calibration, the dependency on the light source is adjusted on-the-fly; we previously introduced this online color calibration for face detection </li></ul><ul><li>M. Rahman, N. Kehtarnavaz, and Jianfeng Ren, “A Hybrid Face Detection Approach For Real-Time Depolyment On Mobile Devices,” Proceedings of IEEE International Conference on Image Processing (ICIP 2009), Cairo, Egypt, Nov. 2009. </li></ul><ul><li>K-means clustering is used to find the most prominent color cluster (black/white can be a dominant color too) in the SIFT detected logo area </li></ul><ul><li>Chrominance values modeled by a Gaussian Mixture Model (GMM) </li></ul><ul><li>Large color areas with high color probability are considered </li></ul><ul><li>Hu moment invariants (7 invariants) are then used to find the logo area by eliminating similar large color areas </li></ul>
  10. 10. Dominant color cluster in the Cb-Cr color space found on-the-fly and modeled by GMM
  11. 11. SIFT matching Dominant color image Detection after moment invariants
  12. 12. Moment invariants used to detect the logo among similar large color areas
  13. 13. Flowchart of our hybrid algorithm
  14. 14. <ul><li>Sub-block processing </li></ul><ul><li>Minimum logo size </li></ul><ul><li>SIFT image scale down </li></ul><ul><li>Memory access </li></ul><ul><li>Lookup table for Gaussian Mixture Model </li></ul>
  15. 15. Table 1. SIFT detection rates for the Samsung logo using different image sizes Table 2. Detection rates and times for different sample logos Table 3. Tracking results per frame with and without using median filtering. SIFT image size Total SIFT Points Number of Matches Detection Time (ms) Detection Rate (%) 160 x 120 115 17 727 98.9 320 x 480 163 34 1531 96.4 640 x 480 238 28 4138 100 Logo Total SIFT Points Number of Matches Detection Rate (%) Detection Time (ms) DHL 101 24 87.6 747 UTD 55 8 94.0 439 IEEE 45 9 98.4 430 Samsung 115 17 98.9 727 National Instruments 71 8 92.2 499 Logo Tracking Rate (%) With Filtering (%) Time (ms) DHL 84.2 98.1 56 UTD 87.6 98.6 47 IEEE 88.2 98.4 53 Samsung 95.9 99.8 55 National Instruments 94.7 99.6 46
  16. 16. IEEE Samsung DHL UTDallas
  17. 17. <ul><li>A computationally efficient logo detection algorithm is developed by combining SIFT for initial detection (~700 ms) and online color based detection for subsequent frames (~50 ms) providing an average processing rate of 20 fps on PC platform </li></ul><ul><li>Ongoing work involves porting this algorithm to the OMAP mobile platform and its real-time implementation on this mobile platform </li></ul>