1) Pixels (u,v) in an image are projections of 3D points (X,Y,Z) in the world onto the image plane according to the camera's intrinsic and extrinsic parameters. 2) The process of projecting 3D points into 2D pixels is called forward projection and can be represented by a perspective projection matrix that models the camera's intrinsic properties and its position and orientation relative to the world frame. 3) Intrinsic parameters include focal length, principal point, and pixel scale factors that define the camera's imaging geometry, while extrinsic parameters define the rigid transformation between the world and camera coordinate frames.

1. Robert Collins
CSE486, Penn State
Camera Projection
Lecture summary

2. P=[U,V,W] Pixel frame
World frame
p=[u,v]
𝑇𝑝
𝑊
Hand in Camera
Robot Arm
From Pixels to World Frame
u
v
U
V
W

3. Imaging Geometry
Our image gets digitized
into pixel coordinates (u,v)

4. Forward Projection
We want a mathematical model to describe how 3D World points
get projected into 2D Pixel coordinates.

5. Backward Projection
Note, much of vision concerns trying to derive backward
projection equations to recover 3D scene structure from images
(via stereo or motion)

6. Forward Projection

7. Forward Projection : Basic Perspective Projection
So how do we represent this as a matrix equation?
We need to introduce homogeneous coordinates.

8. Forward Projection : Homogeneous Coordinates
Represent a 2D point (x,y) by a 3D point (x’,y’,z’) by adding a
“fictitious” third coordinate.
By convention, we specify that given (x’,y’,z’) we can
recover the 2D point (x,y) as
Note: (x,y) = (x,y,1) = (2x, 2y, 2) = (k x, ky, k) for any nonzero
k (can be negative as well as positive)

9. Forward Projection : Homogeneous Coordinates
Perspective Matrix Equation
(in Camera Coordinates)

10. Forward Projection
Rigid Transformation (rotation+translation)
between world and camera coordinate systems

11. World to Camera Transformation
Avoid confusion: Pw and Pc are not two different points.
They are the same physical point, described in two different
coordinate systems.
Forward Projection

12. World to Camera Transformation
Forward Projection

13. World to Camera Transformation
Forward Projection
Matrix Form, Homogeneous Coords

14. Forward Projection : Intrinsic Camera Parameters

15. Describes coordinate transformation between film coordinates
(projected image) and pixel array
• Film cameras: scanning/digitization
• CCD cameras: grid of photosensors
Forward Projection : Intrinsic Camera Parameters

16. Forward Projection : Intrinsic Camera Parameters
Offset
ox and oy called image center or principle point

17. Forward Projection : Intrinsic Camera Parameters
Offset
sometimes one or more coordinate axes are flipped

18. Forward Projection : Intrinsic Camera Parameters
Scale
sampling determines how many rows/cols in the image
?? mm/pixel

19. Effective Scales: sx and sy
Forward Projection : Intrinsic Camera Parameters
Note, since we have different scale factors in x and y,
we don’t necessarily have square pixels !
Aspect ratio is sy / sx

20. Perspective projection matrix
Forward Projection : Intrinsic Camera Parameters
Adding the intrinsic parameters into the perspective projection matrix:
To verify:

21. Note:
Sometimes, the image and the camera coordinate systems
have opposite orientations: [the book does it this way]
Perspective projection matrix
Forward Projection : Intrinsic Camera Parameters

22. Perspective projection matrix : Affine Trans
Forward Projection : Intrinsic Camera Parameters
In general, I like to think of the conversion as
a separate 2D affine transformation from film coords (x,y)
to pixel coordinates (u,v):

23. Summary : Forward Projection

24. Summary: Projection Equation