Evaluating color descriptors for object and scene recognition
•Download as PPTX, PDF•
1 like•718 views
Van De Sande, Koen EA, Theo Gevers, and Cees GM Snoek. "Evaluating color descriptors for object and scene recognition." Pattern Analysis and Machine Intelligence, IEEE Transactions on 32.9 (2010): 1582-1596.
Recommended
More Related Content
Viewers also liked
Viewers also liked (19)
Similar to Evaluating color descriptors for object and scene recognition
Similar to Evaluating color descriptors for object and scene recognition (20)
More from SOYEON KIM
More from SOYEON KIM (8)
Recently uploaded
Recently uploaded (20)
Evaluating color descriptors for object and scene recognition
- 1. Evaluation of Color Descriptors for Object and Scene Recognition Koen E.A. van de Sande, Student Member, IE EE, Theo Gevers, Member, IEEE, and Cees G.M. Snoek, Member, IEEE IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELL IGENCE, VOL. 32, NO. 9, SEPTEMBER 2010
- 2. Introduction To increase illumination invariance and discriminative power Color features/descriptors on object and scene reco gnition The usefulness of invariance is category-specific Recommendations on which color descriptors to use under data sets
- 3. Reflectance Model An image f can be modeled under the assum ption of Lambertian reflectance as follows: Shafer proposes adding a diffuse term： Light source Surface reflectance Camera sensitivity
- 4. Reflectance Model The spatial derivative of f at location x on sca le : invariance to diffuse light!
- 5. Diagonal Model Changes in the illumination can be modeled by a diagonal mapping or von Kries Model as follows: 𝑓 𝑐 = 𝐷 𝑢,𝑐 𝑓 𝑢 Diagonal-offset model: 𝑎 0 0 0 𝑏 0 0 0 𝑐 𝑅 𝐺 𝐵 + 𝑜1 𝑜2 𝑜3 unknown light source Same image transformed
- 6. Photometric transforms Light intensity changes Light intensity shifts B G R a a a 00 00 00 3 2 1 o o o B G R Scale-invariant with respect to light intensity Shift-invariant with respect to light intensity
- 7. Photometric transforms Light intensity scale and shift invariant Light color change Light color change and shift 3 2 1 00 00 00 o o o B G R a a a B G R c b a 00 00 00 3 2 1 00 00 00 o o o B G R c b a
- 8. Color Descriptors Histograms don’t contain local spatial informati on. RGB, Hue, Saturation, rgHistogram, … Color Moments contain local photometrical and spatial information. SIFT contain local spatial information. Color SIFT combined color and SIFT HSV-SIFT, Hue-SIFT, … dxdyyxIyxIyxIyxM c B b G a R qpabc pq )],([)],([)],([
- 9. Color Histograms RGB-histogram Hue-histogram H and S are scale-invariant and shift-invariant w.r.t light intensity rg-histogram The normalized RGB color model r,g Scale-invariant (b is redundan t) Not shift-invariant BGR B BGR G BGR R b g r
- 10. Color Histograms Transformed color Normalized the pixel value distri butions Scale and shift-invariant w.r.t lig ht intensity. Opponent color histogram O1,O2 shift invariant O3: intensity, no invariant B B G G R R B G R B G R 3 6 2 2 3 2 1 BGR BGR GR O O O
- 11. Color SIFT Descriptors HSV-SIFT H color model is scale-invariant and shift-variant Complete descriptor have no invariance properties due to the combination of the HSV channels Hue-SIFT Concatenation of the hue histogram with SIFT Scale-invariant & shift-invariant
- 12. Color SIFT Descriptors OpponentSIFT SIFT over all channels in the opponent color space. Scale & shift Invariant to light intensity C-SIFT Eliminate O1 and O2’s intensity information Scale-invariant to light intensity rg-SIFT SIFT over r,g spaces Scale and shift invariant to light intensity 3 2 3 1 2 1 O O O O O O
- 13. Color SIFT Descriptors RGB-SIFT(Transformed color SIFT) SIFT over every RGB channel (normalized transf ormed channels) Scale- and shift-invariant to light color changes a nd shift.
- 14. Experiments Scale-invariants points by Harris-Laplace point detectors Color descriptors are computed over the area around the points By applying K-means clustering to descriptors, visual dictionary is constructed SVM classifier with EMD/chi-square kernel
- 16. 16
- 17. 17
- 19. 19 RESULTS : Experiment1 The SIFT and color SIFT descriptors perform much better than histogram-based descripto rs The descriptors with the best overall perfor mance are C-SIFT, rgSIFT, OpponentSIFT, an d RGB-SIFT.
- 20. RESUTLS : Expreiment2 Image: PASCAL VOC 2007, over 20 object cat egories
- 22. RESUTLS : Expreiment2 Most objs were categorized better under sca le- and shift- invariant to light intensity C-SIFT, rgSIFT performed better than other o nes The additional invariance makes the descript or less discriminative for these object catego ries because a reduction in performance is o bserved.
- 23. RESUTLS : Expreiment3 Video: Mediamill Challenge, 39 object and sc ene categories
- 24. RESUTLS : Expreiment3 SIFT and color SIFT variants perform significa ntly better than the other descriptors. OpponentSIFT perform better than C-SIFT an d rgSIFT for these categories that occur und er a wide range of light intensities.
- 25. Conclusion A color descriptor with an appropriate level of invariance shou ld be selected Without prior knowledge, OpponentSIFt is the best in general Light intensity info. Is important for some categories Usefulness of invariance is category-specific.
Editor's Notes
- E(λ) : the color of the light source. ρk(λ) ： the camera sensitivity function (k →{R,G,B}). S(x, λ)：the surface reflectance W：the visible spectrum x ：the and the spatial coordinates A(λ)：the term that models the diffuse light.
- F u：the image taken under an unknown light source. F c：the same image transformed, so it appears as if it was taken under the reference light (called canonical illuminant). D：a diagonal matrix which maps colors that are taken under an unknown light source u to their corresponding colors under the canonical illuminant c
- 1. change by a constant factor in all channels (i.e., a=b=c), equal to a light intensity change (light intensity changes also include (no-colored) shadows and shading.) 2. due to diffuse lighting, including scattering of a white light source, object highlights (specular component of the surface) under a white light source, interreflections, and infrared sensitivity of the camera sensor.