Camera model and imaging geometry

1. Camera Model and Imaging Geometry
Subject: Image Procesing & Computer Vision
Dr. Varun Kumar
Subject: Image Procesing & Computer Vision Dr. Varun Kumar (IIIT Surat)Lecture 7 1 / 17

2. Outlines
1 Relationship between Cartesian and Homogeneous Coordinate System
2 Perspective Transformation and Imaging Process
3 Learning of Imaging Geometry
4 References
Subject: Image Procesing & Computer Vision Dr. Varun Kumar (IIIT Surat)Lecture 7 2 / 17

3. Image Formation
Here, (x, y, 0)− − − − − − −
λ
(0, 0, λ)− − − − − − −
Z−λ
Z
(X, Y , Z)
(x, y, z) → Camera co-ordinate system.
(X, Y , Z) → World co-ordinate system aligned with camera
co-ordinate system.
(0, 0, λ) → Center of the lens.
Subject: Image Procesing & Computer Vision Dr. Varun Kumar (IIIT Surat)Lecture 7 3 / 17

4. Continued–
Aim of the camera
Projection of (X, Y , Z) on the image plane → x-y plane.
Homogeneous co-ordinates
x
λ = − X
Z−λ = X
λ−Z ⇒ x = λX
λ−Z
y
λ = − Y
Z−λ = Y
λ−Z ⇒ y = λY
λ−Z
z
λ = − Z
Z−λ = Z
λ−Z ⇒ z = λZ
λ−Z → No need for discussion
Cartesian co-ordinates Homogeneous co-ordinates
(X, Y , Z) ⇒ (kX, kY , kZ, k)
Note: There may be a question.
W =


X
Y
Z

 ⇒ Wh =




kX
kY
kZ
k




Subject: Image Procesing & Computer Vision Dr. Varun Kumar (IIIT Surat)Lecture 7 4 / 17

5. Perspective transform
Perspective transform matrix
Q. How can we find the world co-ordinates from the camera co-ordinate
system that contain the 2D image information?
P =




1 0 0 0
0 1 0 0
0 0 1 0
0 0 −1
λ 1



 ⇒ ch = Pwh =




1 0 0 0
0 1 0 0
0 0 1 0
0 0 −1
λ 1








kX
kY
kZ
k



 =




kX
kY
kZ
−k Z
λ + k




Here, ch → Camera co-ordinate system in homogeneous form
wh → World co-ordinate system in homogeneous form
Representation in Cartesian co-ordinate
c =


x
y
z

 =


λX/(λ − Z)
λY /(λ − Z)
λZ/(λ − Z)


Subject: Image Procesing & Computer Vision Dr. Varun Kumar (IIIT Surat)Lecture 7 5 / 17

6. Inverse perspective transform
Maps an image point back to 3D
wh = P−1
ch where P−1
=




1 0 0 0
0 1 0 0
0 0 1 0
0 0 1
λ 1




For an image point (x0, y0, 0)
ch =




kx0
ky0
0
k



 ⇒ wh =




kx0
ky0
0
k



 but w =


X
Y
Z

 =


x0
y0
0


This result is unacceptable, because for every value Z in 3D gives 0 value
in its projection.
Subject: Image Procesing & Computer Vision Dr. Varun Kumar (IIIT Surat)Lecture 7 6 / 17

7. Continued–
Equation of the line of world co-ordinate system
X =
x0(λ − Z)
λ
Y =
y0(λ − Z)
λ
Inverse perspective transformation is formulated by using Z component of
ch as a free variable.
ch =




kx0
ky0
kz
k



 ⇒ wh = P−1
ch =




kx0
ky0
kz
kz
λ + k




Cartesian co-ordinate
w =


X
Y
Z

 =


λx0/(λ + z)
λy0/(λ + z)
λz/(λ + z)

 ⇒ X =
λx0
(λ + z)
, Y =
λy0
(λ + z)
, Z =
λz
(λ + z)
Subject: Image Procesing & Computer Vision Dr. Varun Kumar (IIIT Surat)Lecture 7 7 / 17

8. Camera and world co-ordinate system are not aligned
From above expression,
z =
λZ
λ − Z
⇒ X =
x0(λ − Z)
λ
, Y =
y0(λ − Z)
λ
Note: Inverse perspective transform needs knowledge of at least one of
the world co-ordinate point.
3D camera set up
Subject: Image Procesing & Computer Vision Dr. Varun Kumar (IIIT Surat)Lecture 7 8 / 17

9. Continued–
Set up specification
Camera is mounted on a Gimble
Pan angle θ and tilt angle α
Displacement of Gimble from the center (origin) is W0
Displacement of image plane center from Gimble center is r.
Q. A point W in world co-ordinate system is given, how will we find the
corresponding image point.
Ans There are several steps,
Bring the world and co-ordinate system under alignment through set of
transformation.
Apply perspective transform to obtain the image co-ordinate for any
world co-ordinate.
Subject: Image Procesing & Computer Vision Dr. Varun Kumar (IIIT Surat)Lecture 7 9 / 17

10. Transformation steps
Transformation steps
Displacement of gimble center from the origin by W0.
Tilt of x-axis by θ.
Pan of z-axis by α.
Displacement of image plane with respect to gimble center by r.
Subject: Image Procesing & Computer Vision Dr. Varun Kumar (IIIT Surat)Lecture 7 10 / 17

11. Continued–
Transform matrix by Gimble (Translation)
G =




1 0 0 − X0
0 1 0 − Y0
0 0 1 − Z0
0 0 0 1



 , Rθ =




cos(θ) sin(θ) 0 0
−sin(θ) cos(θ) 0 0
0 0 1 0
0 0 0 1




Subject: Image Procesing & Computer Vision Dr. Varun Kumar (IIIT Surat)Lecture 7 11 / 17

12. Tilt operation
Tilt operation
Rα =




1 0 0 0
0 cos(α) sin(α) 0
0 − sin(α) cos(α) 0
0 0 0 1




Subject: Image Procesing & Computer Vision Dr. Varun Kumar (IIIT Surat)Lecture 7 12 / 17

13. Continued–
Concatenation
Two rotation matrices can be concatenated by a single matrix
R = RθRα =




cos(θ) sin(θ) 0 0
−sin(θ)cos(α) cos(θ)cos(α) sin(α) 0
sin(θ)sin(α) − cos(θ)sin(α) cos(α) 0
0 0 0 1




When camera center displaced by r
T =




1 0 0 − r1
0 1 0 − r2
0 0 1 − r3
0 0 0 1




Subject: Image Procesing & Computer Vision Dr. Varun Kumar (IIIT Surat)Lecture 7 13 / 17

14. Continued–
Modified relation between ch and wh
A homogeneous world point wh will be mapped to a homogeneous image
point ch as by given transform
ch = PTRGwh (1)
P → Perspective transform
Cartesian co-ordinates from world co-ordinates
x = λ
(X − X0)cos(θ) + (Y − Y0)cos(θ) − r1
−(X − X0)sin(θ)sin(α) + (Y − Y0)cos(θ)sin(α) − (Z − Z0)cos(α) + r2 + λ
and
y = λ
−(X − X0)sin(θ)cos(α) + (Y − Y0)cos(θ)cos(α) + (Z − Z0)sin(α) − r2
−(X − X0)sin(θ)sin(α) + (Y − Y0)cos(θ)sin(α) − (Z − Z0)cos(α) + r2 + λ
Subject: Image Procesing & Computer Vision Dr. Varun Kumar (IIIT Surat)Lecture 7 14 / 17

15. Example
T =




1 0 0 0
0 1 0 0
0 0 1 −1
0 0 0 1



 , Rθ =




cos(135o
) sin(135o
) 0 0
−sin(135o
) cos(135o
) 0 0
0 0 1 0
0 0 0 1




Rα =




1 0 0 0
0 cos(135o
) sin(135o
) 0
0 − sin(135o
) cos(135o
) 0
0 0 0 1




Subject: Image Procesing & Computer Vision Dr. Varun Kumar (IIIT Surat)Lecture 7 15 / 17

16. Continued–
Basic description
Let center of the camera is λ = 0.055 unit from the image plane.
From above Figure, World co-ordinates of corner point → (1, 1, 0.2)




ˆx
ˆy
ˆz
1



 = RαRθT




1
1
0.2
1



 =




0
0.43
1.55
1




⇒ x = λˆx
λ−ˆz = 0 and y = λˆy
λ−ˆz = −0.0099
Home work question :
Q. What is concatenated transformation matrix for a translation [2,3]
followed by rotation by an angle 45o
Subject: Image Procesing & Computer Vision Dr. Varun Kumar (IIIT Surat)Lecture 7 16 / 17

17. References
M. Sonka, V. Hlavac, and R. Boyle, Image processing, analysis, and machine vision.
Cengage Learning, 2014.
D. A. Forsyth and J. Ponce, “A modern approach,” Computer vision: a modern
approach, vol. 17, pp. 21–48, 2003.
L. Shapiro and G. Stockman, “Computer vision prentice hall,” Inc., New Jersey,
2001.
R. C. Gonzalez, R. E. Woods, and S. L. Eddins, Digital image processing using
MATLAB. Pearson Education India, 2004.
Subject: Image Procesing & Computer Vision Dr. Varun Kumar (IIIT Surat)Lecture 7 17 / 17