About Perception and Hue Histograms in HSV Space

1. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing About Perception and HueAbout Perception and Hue Histograms in HSV SpaceHistograms in HSV Space Michel Alves dos Santos May, 2015 *original *hue-saturation projection *saturation *brightness *hue histogram

2. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing IntroductionIntroduction red channel green channel blue channel What's your perception about the colors of this image? *Barns Grand Teton: Grand Teton National Park Note the marked areas! And the distribution by channel? How each color component behaves in its proper channel?

3. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing IntroductionIntroduction*Barns Grand Teton: Grand Teton National Park Observe the distribution of energy for each channel: But, about the hues? What the RGB model can tell us? This kind of graph shows us how the energy of each channel is distributed! red channel green channel blue channel

4. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing IntroductionIntroduction*Barns Grand Teton: Grand Teton National Park Firstly, we will use the HSV Color Space ... How to build a hue histogram? Now, note the diagram below: Can you identify hues distinctly? Probably not! Because of clusters and the intersection of colors! A histogram of hues can help us improve our perception!

5. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing RGB Vs HSVRGB Vs HSV (Color Spaces)(Color Spaces) RGB Color Space HSV Color Space *Solids produced using POVRay [http://lib.povray.org/] Simplicity in representation and storage Close to human perception Some point operations are faster... Easy interpolation between colors Invariant to illumination transformsHave lighting and color properties togetherHave lighting and color properties together

6. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing RGB Vs HSVRGB Vs HSV (Color Spaces – a common mistake...)(Color Spaces – a common mistake...) RGB Color Space HSV Color Space *Solids produced using POVRay [http://lib.povray.org/] A cone is commonly adopted but cylindrical representation is more mathematically accurate.

7. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing RGB Vs HSVRGB Vs HSV (Color Spaces – Solids Transition)(Color Spaces – Solids Transition) RGB Color Space HSV Color Space *Solids produced using POVRay [http://lib.povray.org/]

8. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing RGB Vs HSVRGB Vs HSV (Transformation)(Transformation) RGB HSV Transformation isn't linear! Some care should be taken with monochromatic pixels! *[avoid divisions by zero]

9. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing RGB Vs HSVRGB Vs HSV (Transformation)(Transformation) RGB HSV Transformation isn't linear! Alternative Some care should be taken with monochromatic pixels! *[avoid divisions by zero]

10. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing RGB Vs HSVRGB Vs HSV (Color Image Thresholding)(Color Image Thresholding) Find the yellow M&M's! A simple task for a human observer, but not so easy for machines: Complexity depends of the space! Using RGB: R: G: B: 200 255 200 255 0 50 Using HSV:(3 intervals) (1 interval)

11. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing RGB Vs HSVRGB Vs HSV (Color Image Thresholding)(Color Image Thresholding) Find the yellow M&M's! A simple task for a human observer, but not so easy for machines: Complexity depends of the space! Using RGB: R: G: B: 200 255 200 255 0 50 Using HSV:(3 intervals) (1 interval) [This approach is independent of saturation or brightness]

12. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing *Barns Grand Teton: Grand Teton National Park RGB Vs HSVRGB Vs HSV (Color Image Thresholding)(Color Image Thresholding) *hues of sky *hues of barn *hues of grass Using this feature some tasks can be easily performed! As detection of regions or objects by its characteristic colors! It's one of the bases of color perception!

13. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing RGB Vs HSVRGB Vs HSV (Color Perception)(Color Perception) And now, what's your perception about the colors of this image? *Barns Grand Teton: Grand Teton National Park Where we have more blue hues? Where we have more orange hues? Where we have more green hues? Here we have a blend of yellow and green hues! Do not fool yourself! (Grass???) (Sky) (Barn) *hue histogram *hue projection

14. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing RGB Vs HSVRGB Vs HSV (Color Perception)(Color Perception) And now, what's your perception about the colors of this image? *Barns Grand Teton: Grand Teton National Park Where we have more blue hues? Where we have more orange hues? Where we have more green hues? Here we have a blend of yellow and green hues! Do not fool yourself! (Grass???) (Sky) (Barn) *hue histogram *hue projection

15. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing RGB Vs HSVRGB Vs HSV (Meanings of the Components)(Meanings of the Components) Energy of red channel Energy of green channel Energy of blue channel What is the basic color? How vibrant is the color? How brightness is the color?

16. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing Hue HistogramsHue Histograms *hue projection *saturation *value We will build our histograms taking account the number of positions of the hue circle! Number of bins (discrete positions): 0..360 or 0 to 2π Note that the hue channel is a ring structure!

17. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing Hue Histograms - StepsHue Histograms - Steps Our input images are in RGB Color Space, the output is a transformed hue histogram... We are applying a square root transformation to obtain the final hue histogram... Observe the follow diagram! The metric of contribution is a flexible part of the process and depends of application...

18. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing Hue Histograms - StepsHue Histograms - Steps Our input images are in RGB Color Space, the output is a transformed hue histogram... We are applying a square root transformation to obtain the final hue histogram... Observe the follow diagram! The metric of contribution is a flexible part of the process and depends of application... Get to work! Now, I will show some metrics!

19. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing Hue Histograms :: Simple FrequencyHue Histograms :: Simple Frequency*Barns Grand Teton: Grand Teton National Park Kronecker Delta Hue Picture or Image *hue histogram This is a simple counting! It doesn't consider other HSV channels. Bin Position PixelHistogram Notation

20. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing Hue Histograms :: by SaturationHue Histograms :: by Saturation*Barns Grand Teton: Grand Teton National Park *hue histogram Kronecker Delta Hue Picture Only considers the saturation channel... Saturation Ref: [LP'15] Bin Position PixelHistogram

21. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing Hue Histograms :: by ContributionHue Histograms :: by Contribution*Barns Grand Teton: Grand Teton National Park *hue histogram Kronecker DeltaSaturation ValuePicture Hue Histogram Pixel Bin Position

22. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing Hue Histograms ::Hue Histograms :: Weighted by ContributionWeighted by Contribution *Barns Grand Teton: Grand Teton National Park *hue histogram Kronecker DeltaSaturation ValuePicture Hue *similar to the previous! Normalization Factor Bin Position Pixel

23. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing Hue Histograms :: Hasan WeightedHue Histograms :: Hasan Weighted Ref: [MAH'12] *hue histogram *Barns Grand Teton: Grand Teton National Park Picture Previous Histogram *exponential weighted histogram Mean of Hues *similar to the previous??? Mean Variance Variance of Hues Notation Weighted Histogram Bin Position

24. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing Hue Histograms :: Hasan WeightedHue Histograms :: Hasan Weighted Ref: [MAH'12] *hue histogram *Barns Grand Teton: Grand Teton National Park Picture Previous Histogram *exponential weighted histogram Mean of Hues *similar to the previous??? Variance of Hues *before weighting *after weighting frequency frequencyhue hue Weighted Histogram Bin Position

25. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing Hue Histograms :: Gaussian WeightedHue Histograms :: Gaussian Weighted Gaussian Function *Barns Grand Teton: Grand Teton National Park *hue histogram Picture Previous Histogram Weighted Histogram Bin Position Mean of Hues Variance of Hues

26. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing Hue Histograms ::Hue Histograms :: Smoothed by BoxSmoothed by Box *hue histogram, window size = 1x9 *Barns Grand Teton: Grand Teton National Park Box function Convolution Gaussian Function Previous HistogramWeighted Histogram

27. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing Hue Histograms ::Hue Histograms :: Smoothed by BoxSmoothed by Box *hue histogram, window size = 1x9 *Barns Grand Teton: Grand Teton National Park Box function Convolution Gaussian Function Previous HistogramWeighted Histogram *smoothing employing a boxcar function

28. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing Hue Histograms ::Hue Histograms :: Smoothed by GaussianSmoothed by Gaussian *hue histogram, window size = 1x9 *Barns Grand Teton: Grand Teton National Park Gaussian function Convolution Gaussian Function Previous HistogramWeighted Histogram

29. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing Hue Histograms ::Hue Histograms :: Smoothed by a KernelSmoothed by a Kernel *hue histogram, window size = 1x11 *Barns Grand Teton: Grand Teton National Park kernel Kernel function Convolution Gaussian Function Previous HistogramWeighted Histogram

30. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing Hue Histograms ::Hue Histograms :: Smoothed by a KernelSmoothed by a Kernel *hue histogram, window size = 1x11 *Barns Grand Teton: Grand Teton National Park kernel function employed: Epanechnikov kernel Kernel function Convolution Gaussian Function Previous HistogramWeighted Histogram

31. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing Hue Histograms ::Hue Histograms :: Spatial Color CoherenceSpatial Color Coherence *hue histogram, window size = 1x11 *Barns Grand Teton: Grand Teton National Park kernel Spatial Color Coherence Neighborhood Difference Kernel functionConvolution *8-connected neighborhood Gaussian FunctionWeighted Histogram Saturation Value Kronecker Delta

32. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing Hue Histograms ::Hue Histograms :: Spatial Color CoherenceSpatial Color Coherence *hue histogram, window size = 1x11 *Barns Grand Teton: Grand Teton National Park kernel Spatial Color Coherence Neighborhood Difference Kernel functionConvolution *8-connected neighborhood Gaussian FunctionWeighted Histogram Saturation Value Kronecker Delta *Spatial Color Coherence Map

33. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing Hue Histograms :: The end of our jorney...Hue Histograms :: The end of our jorney... Simple Frequency By Contribution Weighted Contribution Hasan Weighted By Saturation Gaussian WeightedSmoothed A brief visualA brief visual comparison!comparison! This is the end!

34. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing ConclusionConclusion Use the appropriate space for your problem! Always! Hue based histograms can be employed in a wide range of applications! Color spaces that have hue separation approximate human perception! Some considerations should be taken into account in the construction of a hue histogram as brightness and perceptive contribution...

35. Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/Laboratory of Computer Graphics :: March/2015 Michel Alves -- Five Minute Speech (FMS) :: Overview of Activities Developed in Disciplines and Guided Studies :: Graduate Program in Systems Engineering and Computing Thanks for your attention!Thanks for your attention! Michel Alves dos SantosMichel Alves dos Santos References:References: Digital Image Processing (3rd Ed), Gonzalez & Woods, Pages 407-437. [GW'08] Blind Stain Decomposition for Histo-Pathology Images Using Circular Nature of Chroma Components, Li & Plataniotis, 2015, Page 2. [LP'15] Color Image Segmentation Using Acceptable Histogram Segmentation, Delon et al., 2005, LNCS. Shot Classification Using Domain Specific Features for Movie Management, Hasan et al., 2012, LNCS, Springer. [MAH'12] Contrast Enhancement of Color Images Based on Wavelet Transform and Human Visual System, 2007, Xiao & Ohya, IASTED. Progressive Histogram Reshaping for Creative Color Transfer and Tone Reproduction, Pouli & Reinhard, 2010, NPAR2010.