The document discusses camera projection and imaging geometry. It describes how 3D world points are projected onto a 2D image plane through a camera. This process involves multiple coordinate transformations that can be represented by matrix equations. Specifically, it involves a perspective projection from world to camera coordinates, followed by intrinsic camera calibration to map from the camera coordinate system to pixel coordinates on the image sensor.

1. Robert Collins
CSE486, Penn State
Lecture 13:
Camera Projection II
Reading: T&V Section 2.4

2. Robert Collins
CSE486, Penn State
Recall: Imaging Geometry
W
Object of Interest
in World Coordinate
System (U,V,W)
V
U

3. Robert Collins
CSE486, Penn State
Imaging Geometry
Camera Coordinate
Y System (X,Y,Z).
X Z
• Z is optic axis
f • Image plane located f units
out along optic axis
• f is called focal length

4. Robert Collins
CSE486, Penn State
Imaging Geometry
W
Y y
X Z x
V
U
Forward Projection onto image plane.
3D (X,Y,Z) projected to 2D (x,y)

5. Robert Collins
CSE486, Penn State
Imaging Geometry
W
Y y
X Z x
V
u U
v Our image gets digitized
into pixel coordinates (u,v)

6. Robert Collins
CSE486, Penn State
Imaging Geometry
Camera Image (film) World
Coordinates Coordinates W
Coordinates
Y y
X Z x
V
u U
v Pixel
Coordinates

7. Robert Collins
CSE486, Penn State
Forward Projection
World Camera Film Pixel
Coords Coords Coords Coords
U X x u
V Y y v
W Z
We want a mathematical model to describe
how 3D World points get projected into 2D
Pixel coordinates.
Our goal: describe this sequence of
transformations by a big matrix equation!

8. Robert Collins
CSE486, Penn State
Intrinsic Camera Parameters
World Camera Film Pixel
Coords Coords Coords Coords
U X x u
V Y y v
W Z
Affine Transformation

9. Robert Collins
CSE486, Penn State
Intrinsic parameters
• Describes coordinate transformation
between film coordinates (projected image)
and pixel array
• Film cameras: scanning/digitization
• CCD cameras: grid of photosensors
still in T&V section 2.4

10. Robert Collins
CSE486, Penn State
Intrinsic parameters (offsets)
film plane
pixel array
(projected image)
ox (0,0) u (col)
oy x v (row)
(0,0)
y
X Y
u  f  ox v  f  oy
Z Z
ox and oy called image center or principle point

11. Robert Collins
CSE486, Penn State
Intrinsic parameters
sometimes one or more coordinate axes are flipped (e.g. T&V section 2.4)
film plane pixel array
ox (0,0) u (col)
oy y
v (row)
x
(0,0)
X Y
u  f  ox v  f  oy
Z Z

12. Robert Collins
CSE486, Penn State
Intrinsic parameters (scales)
sampling determines how many rows/cols in the image
film scanning
resolution
pixel array
C cols x R rows
CCD
analog
resample

13. Robert Collins
CSE486, Penn State
Effective Scales: sx and sy
1 X 1 Y
u s f  ox v  f  oy
x Z sy Z
Note, since we have different scale factors in x and y,
we don’t necessarily have square pixels!
Aspect ratio is sy / sx
O.Camps, PSU

14. Robert Collins
CSE486, Penn State
Perspective projection matrix
Adding the intrinsic parameters into the
perspective projection matrix:
X 
 x'  f / s x 0 ox 0  
 y '   0 f / sy oy Y 
0
   Z 
 z '  0
   0 1 0  
 1
 
To verify:
x’
u 1 X 1Y
z’ u s f  ox v  f  oy
y’ x Z sy Z
v
z’
O.Camps, PSU

15. Robert Collins
CSE486, Penn State
Note:
Sometimes, the image and the camera coordinate systems
have opposite orientations: [the book does it this way]
X X 
f  ( u  o x ) s x  x'  f / s x 0  ox 0  
Z  y '   0  f / sy  oy Y 
0
Y    Z 
f  ( v  o y )s y  z'  0
   0 1 0  
 1
Z  

16. Robert Collins
CSE486, Penn State
Note 2
In general, I like to think of the conversion as
a separate 2D affine transformation from film
coords (x,y) to pixel coordinates (u,v):
X 
u’  ' 13
a11 a12 xa f 0 0 0  
v’  ya   0
a21 a22 ' 23 f Y 
0 0
w’
   Z 
0 0 z1
 '  0
  0 1 0  
 1
 
Maff Mproj
u = Mint PC = Maff Mproj PC

17. Robert Collins
CSE486, Penn State
Huh?
Did he just say it was “a fine” transformation?
No, it was “affine” transformation, a type of
2D to 2D mapping defined by 6 parameters.
More on this in a moment...

18. Robert Collins
CSE486, Penn State
Summary : Forward Projection
World Camera Film Pixel
Coords Coords Coords Coords
U X x u
V Mext Y Mproj Maff
y v
W Z
U Mext X Mint u
V Y v
W Z
U u
M
V m11 m12 m13 m14 v
W m21 m22 m23 m24
m31 m31 m33 m34

19. Robert Collins
CSE486, Penn State
Summary: Projection Equation
Film plane Perspective
World to camera
to pixels projection
Maff Mproj Mext
Mint
M

20. Robert Collins
CSE486, Penn State
Lecture 13/14:
Intro to Image Mappings

21. Robert Collins
CSE486, Penn State
Image Mappings
Overview
from R.Szeliski

22. Robert Collins
CSE486, Penn State
Geometric Image Mappings
Geometric
image transformation transformed image
(x,y)
(x’,y’)
x’ = f(x, y, {parameters})
y’ = g(x, y, {parameters})

23. Robert Collins
CSE486, Penn State
Linear Transformations
(Can be written as matrices)
Geometric
image transformation transformed image
(x,y)
(x’,y’)
x’ x
y’ = M(params) y
1 1

24. Robert Collins
CSE486, Penn State
Translation
y y’
transform
1
ty
0
0 1 x tx x’
equations matrix form

25. Robert Collins
CSE486, Penn State
Scale
y y’
transform S
1
0 0
0 1 x 0 S x’
equations matrix form

26. Robert Collins
CSE486, Penn State
Rotation
y y’
transform
1

)
0
0 1 x x’
equations matrix form

27. Robert Collins
CSE486, Penn State
Euclidean (Rigid)
y y’
transform 
)
1
ty
0
0 1 x tx x’
equations matrix form

28. Robert Collins
CSE486, Penn State
Partitioned Matrices
http://planetmath.org/encyclopedia/PartitionedMatrix.html

29. Robert Collins
CSE486, Penn State
Partitioned Matrices
2x1 2x2 2x1 2x1
1x1 1x2 1x1 1x1 matrix form
equation form

30. Robert Collins
Another Example (from last time)
CSE486, Penn State
X r11 r12 r13 tx U
Y r21 r22 r23 ty V
Z r31 r32 r33 tz W
1 0 0 0 1 1
3x1 3x3 3x1 3x1
PC R T PW
1x1 = 1x3 1x1 1x1
1 0 1 1
PC = R P W + T

31. Robert Collins
CSE486, Penn State
Similarity (scaled Euclidean)
y y’ S
transform 
)
1
ty
0
0 1 x tx x’
equations matrix form

32. Robert Collins
CSE486, Penn State
Affine
y y’
transform
1
0
0 1 x x’
equations matrix form

33. Robert Collins
CSE486, Penn State
Projective
y y’
transform
1
0
0 1 x x’
Note!
equations matrix form

34. Robert Collins
Summary of 2D Transformations
CSE486, Penn State

35. Robert Collins
Summary of 2D Transformations
CSE486, Penn State
Euclidean

36. Robert Collins
Summary of 2D Transformations
CSE486, Penn State
Similarity

37. Robert Collins
Summary of 2D Transformations
CSE486, Penn State
Affine

38. Robert Collins
Summary of 2D Transformations
CSE486, Penn State
Projective

39. Robert Collins
CSE486, Penn State
Summary of 2D Transformations
from R.Szeliski