Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Robert CollinsCSE486                 Lecture 27:                 Skin Color
Robert CollinsCSE486                   Review: Light Transport    Source emits photons                                    ...
Robert CollinsCSE486                      Color of Light Source                                             Relative amoun...
Robert CollinsCSE486                             Spectral Albedo     Ratio of incoming to outgoing radiation at different ...
Robert CollinsCSE486                         Spectral Radiance                                                   to a     ...
Robert CollinsCSE486                 Human Eye: Rods and Cones                         rods (overall intensity)           ...
Robert CollinsCSE486                 Putting it all Together = Color                                         3 cones      ...
Robert CollinsCSE486                       Describing Color           Today we consider a sample material, human skin,    ...
Robert CollinsCSE486                 Goal: Label Skin Pixels in an Image                 Applications:                   P...
Robert CollinsCSE486                      The Physics of Skin Color          Analytic derivation:          Moritz Storring...
Robert CollinsCSE486                   Problem: Color Variation   Apparent color varies due to lighting color and and came...
Robert CollinsCSE486                    Skin Reflectance Model       Skin is well-modeled by a dichromatic reflectance mod...
Robert CollinsCSE486                 Measuring Spectral Albedo of Skin
Robert CollinsCSE486                 Understanding Skin Albedo                                   red                     b...
Robert CollinsCSE486                   Understanding Skin Albedo                 Increase in melanin yields darker skin, m...
Robert CollinsCSE486                             Analytic Model         Generate different skin albedos by using observed ...
Robert CollinsCSE486                                 Illuminant SPD           Blackbody sources                Artificial ...
Robert CollinsCSE486                 Camera Spectral Response                   SONY DXC-755P 3CCD                 (manufa...
Robert CollinsCSE486                 Skin Color Locus : Analytic Computation                                          “Nor...
Robert CollinsCSE486                 Skin Color Locus : Experimental Measure           Fairly good agreement!
Robert CollinsCSE486                      Skin Locus Examples            Histograms of skin color for different lighting  ...
Robert CollinsCSE486                          Tighter Bounds           If you know the camera and light source, you can   ...
Robert CollinsCSE486                            Example        Same individual under different lighting conditions.       ...
Robert CollinsCSE486                      Sample Application        Face tracking under varying illumination conditions
Robert CollinsCSE486                           Jones and Rehg, 2002                 “Statistical Color Models with Applica...
Robert CollinsCSE486                      Learning from Examples         First, have some poor grad student hand label tho...
Robert CollinsCSE486                      Learned Distributions                 Skin color                   P(rgb | skin)...
Robert CollinsCSE486                                Likelihood Ratio                                      P(rgb | skin)   ...
Robert CollinsCSE486                 Sample Pixel Classifications  Θ = .4
Robert CollinsCSE486                 Sample Application: HCI                               Haiying Guan, Matthew          ...
Robert CollinsCSE486                 Sample Application: HCI                  Haiying Guan, Matthew Turk, UCSB
Robert CollinsCSE486                 Sample Use: Adult Image Classification                  Based on Five Features:      ...
Robert CollinsCSE486                 Adult Image Classification
Robert CollinsCSE486                 Combining Color and Text
Robert CollinsCSE486                 Adult Image Classification       Other related work:         M.M. Fleck, D.A. Forsyth...
Robert CollinsCSE486                 Back to Jones and Rehg Model          A compact description is provided by converting...
Robert CollinsCSE486                 Jones and Rehg Mixture Model
Robert CollinsCSE486                 Jones and Rehg Mixture Model
Robert CollinsCSE486                 Homework: Due Friday Dec 7          •Download jrmogskin.m from             the course...
Robert CollinsCSE486                       What to Hand In       A short report, in Angel:        1) one example where it ...
Robert CollinsCSE486                 Examples Working Well
Robert CollinsCSE486                 Example of False Positives
Robert CollinsCSE486                             Examples           Example of           False Negatives          Explanat...
Robert CollinsCSE486                         Important Constraint                 No X-rated images!!!! Keep it clean     ...
Upcoming SlideShare
Loading in …5
×

Lecture27

890 views

Published on

  • Be the first to comment

  • Be the first to like this

Lecture27

  1. 1. Robert CollinsCSE486 Lecture 27: Skin Color
  2. 2. Robert CollinsCSE486 Review: Light Transport Source emits photons And then some reach an eye/camera and are measured. Photons travel in a straight line They hit an object. Some are absorbed, some bounce off in a new direction.
  3. 3. Robert CollinsCSE486 Color of Light Source Relative amount of light Spectral Power Distribution: energy at each wavelength Amplitude Wavelength λ UV Visible IR
  4. 4. Robert CollinsCSE486 Spectral Albedo Ratio of incoming to outgoing radiation at different wavelengths. Spectral albedo for several different leaves
  5. 5. Robert CollinsCSE486 Spectral Radiance to a Spectral Spectral Spectral Irradiance Albedo Radiance
  6. 6. Robert CollinsCSE486 Human Eye: Rods and Cones rods (overall intensity) S cones (blue) M cones (green) L cones (red)
  7. 7. Robert CollinsCSE486 Putting it all Together = Color 3 cones COLOR!
  8. 8. Robert CollinsCSE486 Describing Color Today we consider a sample material, human skin, and look at two approaches to describe the color of skin in order to find it in images. 1) physics-based approach 2) learning-based approach
  9. 9. Robert CollinsCSE486 Goal: Label Skin Pixels in an Image Applications: Person finding/tracking Gesture recognition
  10. 10. Robert CollinsCSE486 The Physics of Skin Color Analytic derivation: Moritz Storring, Hans Andersen and Eric Granum, “Skin Colour Detection under Changing Lighting Conditions,” 7th Symposium on Intelligent Robotics Systems, Coimbra Portugal, July 1999. Experimental measurement: Birgitta Martinkauppi, “Face Colour Under Varying Illumination: Analysis and Applications,” Ph.D. Thesis, Oulu University Press, Oulu Finland, 2002.
  11. 11. Robert CollinsCSE486 Problem: Color Variation Apparent color varies due to lighting color and and camera spectral response. Sample from Oulu Physics-Based Face Database
  12. 12. Robert CollinsCSE486 Skin Reflectance Model Skin is well-modeled by a dichromatic reflectance model. transparent medium (dermis) pigmentations (hemaglobin, melanin) specular reflection (oil on skin) Dichromatic reflectance model
  13. 13. Robert CollinsCSE486 Measuring Spectral Albedo of Skin
  14. 14. Robert CollinsCSE486 Understanding Skin Albedo red blue
  15. 15. Robert CollinsCSE486 Understanding Skin Albedo Increase in melanin yields darker skin, masking the absorbtion band pattern of the hemaglobin.
  16. 16. Robert CollinsCSE486 Analytic Model Generate different skin albedos by using observed curve for caucasian, and calculate the reduction in reflectance due to an increase in melanin (a substance that has a known absorbtion) Simpler approximation: I1(λ) ~ s I2(λ) ; λ = wavelength s = scale factor
  17. 17. Robert CollinsCSE486 Illuminant SPD Blackbody sources Artificial light sources (for theoretical calculations)
  18. 18. Robert CollinsCSE486 Camera Spectral Response SONY DXC-755P 3CCD (manufacturer can supply this)
  19. 19. Robert CollinsCSE486 Skin Color Locus : Analytic Computation “Normalized Color” recall simple scaling of SPD curves
  20. 20. Robert CollinsCSE486 Skin Color Locus : Experimental Measure Fairly good agreement!
  21. 21. Robert CollinsCSE486 Skin Locus Examples Histograms of skin color for different lighting conditions. Red: high values, blue: low values.
  22. 22. Robert CollinsCSE486 Tighter Bounds If you know the camera and light source, you can derive much tighter analytic bounds on skin color.
  23. 23. Robert CollinsCSE486 Example Same individual under different lighting conditions. Subject 1: Caucasian Subject 2: Asian Indian
  24. 24. Robert CollinsCSE486 Sample Application Face tracking under varying illumination conditions
  25. 25. Robert CollinsCSE486 Jones and Rehg, 2002 “Statistical Color Models with Application to Skin Detection”, M. J. Jones and J. M. Rehg, Int. J. of Computer Vision, 46(1):81-96, Jan 2002 General Idea: • Drop the physics. Learn from examples instead. • Learn distributions of skin and nonskin color • Nonparametric distributions: color histograms • Bayesian classification of skin pixels
  26. 26. Robert CollinsCSE486 Learning from Examples First, have some poor grad student hand label thousands of images P(rgb | skin) = number of times rgb seen for a skin pixel total number of skin pixels seen P(rgb | not skin) = number of times rgb seen for a non-skin pixel total number of non-skin pixels seen These statistics stored in two 32x32x32 RGB histograms Skin histogram Non-Skin histogram R R B B G G
  27. 27. Robert CollinsCSE486 Learned Distributions Skin color P(rgb | skin) Non-Skin color P(rgb | not skin)
  28. 28. Robert CollinsCSE486 Likelihood Ratio P(rgb | skin) Label a pixel skin if > Θ P(rgb | not skin) (cost of false positive) P( seeing not skin) Θ = (cost of false negative) P( seeing skin) 0 <= Θ <= 1
  29. 29. Robert CollinsCSE486 Sample Pixel Classifications Θ = .4
  30. 30. Robert CollinsCSE486 Sample Application: HCI Haiying Guan, Matthew Turk, UCSB
  31. 31. Robert CollinsCSE486 Sample Application: HCI Haiying Guan, Matthew Turk, UCSB
  32. 32. Robert CollinsCSE486 Sample Use: Adult Image Classification Based on Five Features: • Percentage of pixels detected as skin. • Average probability of the skin pixels. • Size in pixels of the largest connected component of skin. • Number of connected components of skin. • Percentage of colors with no entries in the skin and non-skin histograms Jones and Rehg
  33. 33. Robert CollinsCSE486 Adult Image Classification
  34. 34. Robert CollinsCSE486 Combining Color and Text
  35. 35. Robert CollinsCSE486 Adult Image Classification Other related work: M.M. Fleck, D.A. Forsyth and C. Bregler, “Finding Naked People,” Proc. European Conf. on Computer Vision, Springer-Verlag, 1996. p. 593-602 James Wang, Jia Li, Gio Wiederhold and Oscar Firschein, “System for Screen Objectionable Images” Computer Communications, Vol 21(15), pp.1355-1369, Elsevier, 1998.
  36. 36. Robert CollinsCSE486 Back to Jones and Rehg Model A compact description is provided by converting the histogram-based model into a Gaussian Mixture model.
  37. 37. Robert CollinsCSE486 Jones and Rehg Mixture Model
  38. 38. Robert CollinsCSE486 Jones and Rehg Mixture Model
  39. 39. Robert CollinsCSE486 Homework: Due Friday Dec 7 •Download jrmogskin.m from the course web site •Try it on your own images!
  40. 40. Robert CollinsCSE486 What to Hand In A short report, in Angel: 1) one example where it works wonderfully well 2) one example showing false positives (things that are not skin, but that are labeled as skin). 3) one example showing false negatives (a patch of skin that is not labeled), along with an educated guess about why it was missed.
  41. 41. Robert CollinsCSE486 Examples Working Well
  42. 42. Robert CollinsCSE486 Example of False Positives
  43. 43. Robert CollinsCSE486 Examples Example of False Negatives Explanation: paint on skin changes the spectral albedo
  44. 44. Robert CollinsCSE486 Important Constraint No X-rated images!!!! Keep it clean for the report.

×