Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Evaluation of how color descriptors recognize objects and scenes
1. Evaluation of Color Descriptors for Object and Scene Recognition Authors: Koen E. A. van de Sande, Theo Gevers, and Cees G. M. Snoek @ University of Amsterdam Presenter: Shao-Chuan Wang
2. Evaluation of Color Descriptors for Object and Scene Recognition Focus: Color features/descriptors on obj. and scene recognition Summary: The invariance of photometric transform aces and its effect on discriminative power. Conclusion: The usefulness of invariance is category-specific!
4. Photometric transforms (1/2) Light intensity scale and shift invariant Light color change Light color change and shift
5. Color Descriptors (1/1) Histograms RGB, Hue, Saturation, rgHistogram, … Color Moments: contain spatial info. Color SIFT: combined color and SIFT HSV-SIFT, Hue-SIFT, …
6. Color Histograms (1/2) RGB-histogram Hue-histogram H and S are scale-invariant and shift-invariant w.r.t light intensity rg-histogram Scale-invariant Not shift-invariant Not that b is redudant Image from wikipedia
7. Color Histograms (2/2) Transformed color Scale and shift-invariant w.r.t light intensity. Opponent color histogram O1,O2 shift invariant O3: intensity, no invariant
8. Color SIFT Descriptors (1/3) HSV-SIFT SIFT over HSV channels Hue is unstable in gray axis Hue-SIFT (Van de Weijer 2006) Used Hue histogram weighing by its saturation Concatenate Hue histogram with SIFT Only SIFT is invariant; Hue histogram is not! (partial invariance) Hue Instability
9. Color SIFT Descriptors (2/3) OpponentSIFT SIFT over all channels in the opponent color space. Shift-invariant to light intensity. W-SIFT Eliminate O1 and O2’s intensity information Scale-invariant to light intensity rg-SIFT SIFT over r,g spaces Scale and shift invariant, but not invariant to light color changes/shifts
10. Color SIFT Descriptors (3/3) Transformed color SIFT SIFT over normalized transformed channels. Scale- and shift-invariant to light color changes and shift.
11. Experiments Implementation: Scale-invariants points are detected by Harris-Laplace point detectors Color descriptors are computed over the area around the points; all regions are proportionally re-sampled to a uniform 60x60 patch. Cluster descriptors with k = 40 (images) k = 4000 (video) SVM classifier with EMD/chi-square kernel
13. Benchmark (2/3) Most objs were categorized better under scale- and shift- invariant to light intensity Some, such as car and dining table, do not benefit from such invariance.
15. Evaluation of Color Descriptors for Object and Scene Recognition Conclusion: W-SIFT and rgSIFT, in general, outperform other color descriptors. Light intensity info. Is important for some categories Usefulness of invariance is category-specific.