1. The document discusses image formation, cameras, and digital image acquisition and representation. It describes how images are formed through light projection and sampling, and how analog and digital cameras work to capture images. 2. Digital images are represented as matrices, with each element corresponding to a pixel value. Grayscale images have a single value per pixel while color images have multiple values representing channels like red, green, and blue. 3. Pixels in digital images are quantized to a finite set of numeric values like 8-bit integers from 0 to 255 for storage and processing in computer systems. This affects qualities like radiometric resolution of the encoded image.

1. Image and Camera
Fundamentals
Elsayed Hemayed

2. Overview
• Image Formation
• Cameras
• Digital Image Acquisition
• Matrix Representation of Images
• Image Characteristics
• Digital Color Images
• Image Display
• Human Eye
2Image & Camera

3. Image Formation
• Camera obscura (dark chamber)
"When images of illuminated objects ... penetrate through a small hole into a very dark room ...
you will see [on the opposite wall] these objects in their proper form and color, reduced in size ...
in a reversed position, owing to the intersection of the rays".
Leonardo Da Vinci 3Image & Camera

4. Figure from US Navy Manual of Basic Optics and Optical Instruments, prepared by Bureau of Naval Personnel. Reprinted by Dover Publications, Inc., 1969.
Images are two-dimensional patterns of brightness values.They are formed by the
projection of 3D objects.
Image Formation
4

5. Animal eye: a looonnng time ago.
Photographic camera:
Niepce, 1816.
Reproduced by permission, the American Society of Photogrammetry and
Remote Sensing. A.L. Nowicki, “Stereoscopy.” Manual of Photogrammetry,
Thompson, Radlinski, and Speert (eds.), third edition, 1966.
Figure from US Navy
Manual of Basic Optics
and Optical Instruments,
prepared by Bureau of
Naval Personnel. Reprinted
by Dover Publications,
Inc., 1969.
Pinhole perspective projection: Brunelleschi, XVth Century.
Camera obscura: XVIth Century. 5

6. Cameras
• It used a pinhole to focus light rays onto a wall or
translucent plate.
• Pinholes were replaced by lenses.
• Light collected over the imaging surface
produces a photograph.
6Image & Camera

7. Pinhole too big -
many directions are
averaged, blurring the
image
Pinhole too small-
diffraction effects blur
the image
Generally, pinhole
cameras are dark, because
a very small set of rays
from a particular point
hits the screen.
Pinhole (Aperture)
size effect
7Image & Camera

8. The reason for lenses
8Image & Camera

9. Digital Image Acquisition
• When photons strike, electron-hole pairs are
generated on sensor sites.
• Electrons generated are collected over a certain
period of time.
• The number of electrons are converted to pixel
values. (Pixel is short for picture element.)
• Pixel values are quantized. For example, in an
8-bit representation the pixel values are integers
in the range [0-255].
Spectral
filter
Lens
Sensor
array
QuantizerExposure
timer
Scene
Digital
image
9

10. The Field of View
10Image & Camera

11. CCD Cameras
charge-coupled-device (CCD)
Shutter Speed: is the length of
time a shutter is open; the total
exposure is proportional to this
exposure time, or duration of light
reaching the film or image sensor.
Focal Length: affects the zoom
Aperture Size: affects the focus
Resolution: affects the details
11Image & Camera
See http://www.engineerguy.com/elements/videos/video-ccd.htm

12. Digital Image Acquisition
12

13. Digital Image Acquisition
Spatial sampling and amplitude quantization 13Image & Camera

14. Quantization Effects –
Radiometric Resolution
8 bit – 256 levels 1 bit – 2 levels2 bit – 4 levels4 bit – 16 levels

15. Matrix Representation of Images
• A digital image can be written as a matrix
1 2
[0,0] [0,1] [0, 1]
[1,0] [1,1] [1, 1]
[ , ]
[ 1,0] [ 1, 1] MxN
x x x N
x x x N
x n n
x M x M N
 
  
 
 
   
35 45 20
43 64 52
10 29 39
 
 
 
  
15Image & Camera

16. What is an image?
16

17. RRf
yxfP


2
:
),(
17

18. RRf
yxfP


2
:
),(
18
1. We sample the
image to get a
discrete set of
pixels with
quantized values.
2. For a gray tone
image there is one
band F(r,c), with
values usually
between 0 and
255.
3. For a color image
there are 3 bands
R(r,c), G(r,c), B(r,c)

19. Images in Python Numpy
N x M grayscale image “im”
– im[0,0] = top-left pixel value
– im[y, x] = y pixels down, x pixels to right
– im[N-1, M-1] = bottom-right pixel
0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.99
0.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.91
0.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.91 0.92
0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.95
0.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.85
0.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33
0.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74
0.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93
0.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99
0.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97
0.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93
Row
Column
James Hays

20. Grayscale intensity
0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.99
0.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.91
0.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.91 0.92
0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.95
0.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.85
0.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33
0.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74
0.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93
0.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99
0.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97
0.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93

21. Color R
G
B
James Hays

22. Images in Python Numpy
N x M RGB image “im”
– im[0,0,0] = top-left pixel value in R-channel
– Im[x, y, b] = x pixels to right, y pixels down in the bth channel
– Im[N-1, M-1, 3] = bottom-right pixel in B-channel
0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.99
0.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.91
0.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.91 0.92
0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.95
0.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.85
0.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33
0.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74
0.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93
0.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99
0.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97
0.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93
0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.99
0.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.91
0.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.91 0.92
0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.95
0.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.85
0.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33
0.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74
0.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93
0.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99
0.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97
0.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93
0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.99
0.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.91
0.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.91 0.92
0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.95
0.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.85
0.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33
0.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74
0.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93
0.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99
0.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97
0.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93
R
G
B
Row
Column
James Hays

23. Images in Python Numpy
Take care between types!
- uint8 (values 0 to 255) – io.imread(“file.jpg”)
- float32 (values 0 to 255) – io.imread(“file.jpg”).astype(np.float32)
- float32 (values 0 to 1) – img_as_float32(io.imread(“file.jpg”))
0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.99
0.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.91
0.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.91 0.92
0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.95
0.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.85
0.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33
0.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74
0.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93
0.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99
0.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97
0.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93
Row
Column
James Hays

24. Image Characteristics
• Image Resolution
• Image Size
• Bit Depth – Grayscale Resolution
24Image & Camera

25. Image Characteristics
Image Resolution
25

26. Image Characteristics
Image Size
26

27. Image Characteristics
8 bits 7 bits
6 bits 5 bits
Bit Depth – Grayscale Resolution
27

28. Image Characteristics
4 bits 3 bits
2 bits 1 bit
Bit Depth – Grayscale Resolution
Contouring
28Image & Camera

29. Color Sensing in Camera (RGB)
• 3-chip vs. 1-chip: quality vs. cost
• Why more green?
http://www.cooldictionary.com/words/Bayer-filter.wikipedia
Why 3 colors?
Slide by Steve Seitz

30. Practical Color Sensing: Bayer Grid
• Estimate RGB
at ‘G’ cells
from
neighboring
values
Slide by Steve Seitz

31. Color spaces
• How can we represent color?
http://en.wikipedia.org/wiki/File:RGB_illumination.jpg

32. Color spaces: RGB
0,1,0
0,0,1
1,0,0
Image from: http://en.wikipedia.org/wiki/File:RGB_color_solid_cube.png
Any color = r*R + g*G + b*B
• Strongly correlated channels
• Non-perceptual
Default color space
R = 1
(G=0,B=0)
G = 1
(R=0,B=0)
B = 1
(R=0,G=0)

33. Most information in intensity
Only color shown – constant intensity
James Hays

34. Most information in intensity
Only intensity shown – constant color
James Hays

35. Most information in intensity
Original image
James Hays

36. HSI
• Hue refers to the perceived color (the dominant
wavelength)- Example: purple
• Saturation measures its dilution by white light. Light
purple vs. dark purple
• HSI decouples the intensity information from the color
36

37. Color spaces: HSV
Intuitive color space
H
(S=1,V=1)
S
(H=1,V=1)
V
(H=1,S=0)
James Hays

38. Color spaces: YCbCr
Y
(Cb=0.5,Cr=0.5)
Cb
(Y=0.5,Cr=0.5)
Cr
(Y=0.5,Cb=0.5)
Y=0 Y=0.5
Y=1
Cb
Cr
Fast to compute, good for
compression, used by TV
James Hays

39. CMY
• The CMY (cyan-magenta-yellow) model is a subtractive
model appropriate to absorption of colors, for example
due to pigments in paints.
• Whereas the RGB model asks what is added to black to
get a particular color, the CMY model asks what is
subtracted from white.
• Subtractive means that color you see on the paper is a
result of adding together the three secondary colors,
filtering out the unwanted color components, and
reflecting only the desired colors.
• Cyan absorbs the red color component, magenta
absorbs the green, and yellow absorbs the blue.
• For example, mixing magenta and yellow together
results in the red color seen on the paper.
39Image & Camera

40. CMY Vs RGB
The figure on the left shows the additive mixing of red, green and blue primaries to form
the three secondary colours yellow (red + green), cyan (blue + green) and magenta (red +
blue), and white ((red + green + blue).
The figure on the right shows the three subtractive primaries, and their pairwise
combinations to form red, green and blue, and finally black by subtracting all three
primaries from white. 40Image & Camera

41. Image Display
• Display
– CRT, LCD, DLP, Plasma, LCOS, D-ILA
– HDTV, display wall
– PDA, cellular phone, Gameboy
– Stereoscopic (3D)
41Image & Camera

42. CRT: Cathode Ray Tube
Mitsubishi WS-55813
Rear Projection CRT
Sony KV34XBR910
Direct View CRT
Future: extinction
42Image & Camera

43. LCD: Liquid Crystal Display
Philips 42FD9954 Flat Screen
LCD Display
Future: bigger, faster, cheaper
43Image & Camera

44. Virtual Wall
NASA Space Shuttle on the display wall, March 1999
44Image & Camera

45. Display on Mobile Devices
cell phone
portable DVD
Gameboy
PDA
45Image & Camera

46. 3D Display
46Image & Camera

47. The Ultimate Display: Virtual
Retinal Display
47Image & Camera

48. The Human Eye
•The iris and the pupil control the amount of light penetrating the eyeball
•The cornea and the lens refract the light to create the retinal image
•The retina where the image is formed.
•Human eye Field of View: 160x135
48Image & Camera
• The human eye is a camera
– Iris - colored annulus with radial muscles
– Pupil - the hole (aperture) whose size is controlled by the iris
– What’s the sensor? photoreceptor cells (rods and cones) in the retina