Image and Video Processing Image Formation and Color Thomas Breuel
light
 
light <ul><li>electromagnetic radiation
properties of both particle and wave
diffraction, interference
different frequencies/wavelengths
narrowband appears as pure color
“white light” = even mixture </li></ul>
what do we perceive? <ul><li>radiance </li><ul><li>energy emitted/received as light (W) </li></ul><li>luminance </li><ul><...
image processing goals <ul><li>prepare images for visual inspection </li><ul><li>based on properties of visual system
replicate performance of visual system </li></ul><li>prepare images for automated analysis </li><ul><li>based on physical ...
solve a physical problem </li></ul></ul>
perception vs physical reality
discounting the illuminant purveslab.net
discounting the illuminant
reasoning purveslab.net sunrise/sunset midday
incandescent light daylight color constancy
 
Even “low-level” perception (color etc.)  interprets images.  Humans do not consciously perceive  physical intensities or ...
definition of color <ul><ul><li>Farbe ist diejenige Gesichtsempfindung eines dem Auge des Menschen strukturlos erscheinend...
Color is a percept that allows humans to distinguish two untextured, flat, uniform, unmoving surface patches from one anot...
trichromatic vision
experimental evidence (1852) <ul><li>experiment </li><ul><li>observers match given color by mixing different numbers of ba...
Maxwell – Helmholtz experiments
sensors
structure of the retina <ul><li>structure </li><ul><li>cones are in the center
rods are in the periphery </li></ul><li>implications... </li></ul><ul><li>lower density of blue receptors
absent from the center of the fovea </li></ul>
retina, rods, cones... <ul><li>structure </li><ul><li>retina is inverted
cones in fovea, rods in periphery
low density of blue, absent from center of fovea </li></ul><li>why? </li><ul><li>no engineer would design such a system (?)
actually, very finely tuned </li><ul><li>densities and curves optimized for fine spatial discrimination in the presence of...
lots of other complex design components </li></ul><li>some historical accidents </li></ul></ul>
direct imaging of photoreceptors live retinal imaging using adaptive optics
spectral sensitivity
maximum sensitivity <ul><li>cones aren't really RGB </li><ul><li>M and L are close together </li></ul><li>CRT RGB </li><ul...
less overlap
peaky spectrum in red </li></ul></ul>
practical applications <ul><li>foundation of </li><ul><li>display devices
printing
color calibration
image processing
digital photography </li></ul><li>significant for </li><ul><li>accessibility (color blindness)
display design </li></ul></ul>
color vs frequency
color perception spectral sensitivity of cones contextual interpretation light spectrum 3D “RGB” vector color percept
chromaticity diagram <ul><li>x,y,z percentages of RGB
trichromatic coefficients
chromaticity = hue+saturation
brightness not visible in diagram </li></ul>
spectrum and color
Newton: spectrum, additivity, hue circle
metamerism vv http://www.visualmill.com/
physiological color space unrealizable because of overlap of response curves
response vv
wavelength vs rgb
more aspects of color
color difference perception
color gamut
color spaces
color spaces <ul><li>RGB – red green blue
CMY, CMYK – cyan magenta yellow (black)
HSI / HSV / HSL – hue saturation intensity...
XYZ – perceptual space
L*a*b* – perceptual space (normalized dist)
YUV / YIQ / YCbCr – TV spaces </li></ul>
3D RGB space
pixel and color values colors for all on/off components
CMY(K) space
HSI space <ul><li>decouples intensity from chromaticity </li></ul>
Upcoming SlideShare
Loading in...5
×

02-image-formation.odp

426

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
426
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

02-image-formation.odp

  1. 1. Image and Video Processing Image Formation and Color Thomas Breuel
  2. 2. light
  3. 4. light <ul><li>electromagnetic radiation
  4. 5. properties of both particle and wave
  5. 6. diffraction, interference
  6. 7. different frequencies/wavelengths
  7. 8. narrowband appears as pure color
  8. 9. “white light” = even mixture </li></ul>
  9. 10. what do we perceive? <ul><li>radiance </li><ul><li>energy emitted/received as light (W) </li></ul><li>luminance </li><ul><li>radiance adjusted by sensitivity of the eye (lm) </li></ul><li>brightness </li><ul><li>perceptual quantity (“which is brighter?”) </li></ul></ul>
  10. 11. image processing goals <ul><li>prepare images for visual inspection </li><ul><li>based on properties of visual system
  11. 12. replicate performance of visual system </li></ul><li>prepare images for automated analysis </li><ul><li>based on physical properties
  12. 13. solve a physical problem </li></ul></ul>
  13. 14. perception vs physical reality
  14. 15. discounting the illuminant purveslab.net
  15. 16. discounting the illuminant
  16. 17. reasoning purveslab.net sunrise/sunset midday
  17. 18. incandescent light daylight color constancy
  18. 20. Even “low-level” perception (color etc.) interprets images. Humans do not consciously perceive physical intensities or light frequencies.
  19. 21. definition of color <ul><ul><li>Farbe ist diejenige Gesichtsempfindung eines dem Auge des Menschen strukturlos erscheinenden Teiles des Gesichtsfeldes, durch die sich dieser Teil bei einäugiger Beobachtung mit unbewegtem Auge von einem gleichzeitig gesehenen, ebenfalls strukturlosen angrenzenden Bezirk allein unterscheiden kann.
  20. 22. Color is a percept that allows humans to distinguish two untextured, flat, uniform, unmoving surface patches from one another by looking at them with one eye. </li></ul></ul>
  21. 23. trichromatic vision
  22. 24. experimental evidence (1852) <ul><li>experiment </li><ul><li>observers match given color by mixing different numbers of base colors </li></ul><li>result </li><ul><li>combinations of three base colors are sufficient </li></ul></ul>
  23. 25. Maxwell – Helmholtz experiments
  24. 26. sensors
  25. 27. structure of the retina <ul><li>structure </li><ul><li>cones are in the center
  26. 28. rods are in the periphery </li></ul><li>implications... </li></ul><ul><li>lower density of blue receptors
  27. 29. absent from the center of the fovea </li></ul>
  28. 30. retina, rods, cones... <ul><li>structure </li><ul><li>retina is inverted
  29. 31. cones in fovea, rods in periphery
  30. 32. low density of blue, absent from center of fovea </li></ul><li>why? </li><ul><li>no engineer would design such a system (?)
  31. 33. actually, very finely tuned </li><ul><li>densities and curves optimized for fine spatial discrimination in the presence of chromatic aberration
  32. 34. lots of other complex design components </li></ul><li>some historical accidents </li></ul></ul>
  33. 35. direct imaging of photoreceptors live retinal imaging using adaptive optics
  34. 36. spectral sensitivity
  35. 37. maximum sensitivity <ul><li>cones aren't really RGB </li><ul><li>M and L are close together </li></ul><li>CRT RGB </li><ul><li>spaced apart further
  36. 38. less overlap
  37. 39. peaky spectrum in red </li></ul></ul>
  38. 40. practical applications <ul><li>foundation of </li><ul><li>display devices
  39. 41. printing
  40. 42. color calibration
  41. 43. image processing
  42. 44. digital photography </li></ul><li>significant for </li><ul><li>accessibility (color blindness)
  43. 45. display design </li></ul></ul>
  44. 46. color vs frequency
  45. 47. color perception spectral sensitivity of cones contextual interpretation light spectrum 3D “RGB” vector color percept
  46. 48. chromaticity diagram <ul><li>x,y,z percentages of RGB
  47. 49. trichromatic coefficients
  48. 50. chromaticity = hue+saturation
  49. 51. brightness not visible in diagram </li></ul>
  50. 52. spectrum and color
  51. 53. Newton: spectrum, additivity, hue circle
  52. 54. metamerism vv http://www.visualmill.com/
  53. 55. physiological color space unrealizable because of overlap of response curves
  54. 56. response vv
  55. 57. wavelength vs rgb
  56. 58. more aspects of color
  57. 59. color difference perception
  58. 60. color gamut
  59. 61. color spaces
  60. 62. color spaces <ul><li>RGB – red green blue
  61. 63. CMY, CMYK – cyan magenta yellow (black)
  62. 64. HSI / HSV / HSL – hue saturation intensity...
  63. 65. XYZ – perceptual space
  64. 66. L*a*b* – perceptual space (normalized dist)
  65. 67. YUV / YIQ / YCbCr – TV spaces </li></ul>
  66. 68. 3D RGB space
  67. 69. pixel and color values colors for all on/off components
  68. 70. CMY(K) space
  69. 71. HSI space <ul><li>decouples intensity from chromaticity </li></ul>
  70. 72. hue, lightness, saturation
  71. 73. decomposing the image to Lab
  72. 74. decomposing the image to Lab
  73. 75. digital cameras
  74. 76. image formation in cameras <ul><li>geometry
  75. 77. optics
  76. 78. sensor
  77. 79. signal processing </li></ul>
  78. 80. pinhole camera model projection, translation, rotation camera equation often write this as matrix multipliations in homogeneous coordinates
  79. 82. color and surfaces n illuminant surface reflectance lens retina, film
  80. 83. The light that arrives at the eye/camera is the product of the spectrum of the illuminant and the surface reflectance of the object (at each wavelength)
  81. 84. image formation model
  82. 85. digital camera sensors
  83. 86. CCD vs CMOS linear response logarithmic response
  84. 87. color sensors (Bayer pattern) <ul><li>higher resolution in green (like human eye)
  85. 88. output in RAW format
  86. 89. final image interpolated (demosaicing) </li></ul>
  87. 90. Kodak KAI-11000 CCD 4008x2672 pixels (11MP)
  88. 91. KAI-11000 quantum efficiency
  89. 92. digitization
  90. 93. digitization <ul><li>spatial quantization </li><ul><li>continuous coordinate system
  91. 94. image plane divided into buckets (pixels) </li></ul><li>intensity quantization </li><ul><li>photon counting (some specialized sensors)
  92. 95. discretization of voltage </li></ul></ul>
  93. 96. intensity quantization
  94. 97. quantization
  95. 98. spatial digitization
  96. 99. sampling vs digitization sampling digitization (equivalent to convolution with a square pillbox, followed by sampling)
  97. 100. quantization <ul><li>uniform quantization </li><ul><li>digital version of analog quantity </li></ul><li>gamma-corrected / logarithmic / non-uniform </li><ul><li>try to get more resolution for “interesting values”
  98. 101. mimic human JNDs
  99. 102. humans “see logarithmically” </li></ul><li>binarization </li><ul><li>special quantization for document images, industrial vision </li></ul></ul>
  100. 103. effect of digitization
  101. 104. effects of quantization
  102. 105. iso-preference curves <ul><li>vary resolution (N) and quantization (k)
  103. 106. ask for preference between versions </li></ul>
  104. 107. image storage
  105. 108. image storage raster storage convention (used by SciPy display/load/save)
  106. 109. image formats <ul><li>JPEG </li><ul><li>lossy compression, natural images </li></ul><li>PNG </li><ul><li>lossless compression, graphics </li></ul><li>TIFF </li><ul><li>multipage, extensible >8bpp, document images </li></ul><li>GIF </li><ul><li>graphics, palette, animation </li></ul><li>JPEG2000 </li><ul><li>lossy wavelet compression, >8bpp </li></ul></ul>
  107. 110. storage <ul><li>raw size </li><ul><li>12 Mpixel, RGB, 8bpp
  108. 111. same in floating point? double? </li></ul><li>factors </li><ul><li>image type (natural, graphics, text, ...)
  109. 112. compression method </li></ul></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×