Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
×

# 14 cv mil_the_pinhole_camera

781 views

Published on

Published in: Business, Technology
• Full Name
Comment goes here.

Are you sure you want to Yes No
Your message goes here
• Be the first to comment

• Be the first to like this

### 14 cv mil_the_pinhole_camera

1. 1. Computer vision: models, learning and inference Chapter 14 The pinhole camera Please send errata to s.prince@cs.ucl.ac.uk
2. 2. Structure• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems – Exterior orientation problem – Camera calibration – 3D reconstruction• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 2
3. 3. MotivationSparse stereo reconstructionCompute the depth at a setof sparse matching points Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 3
4. 4. Pinhole cameraReal camera image Instead model impossible but more is inverted convenient virtual image Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 4
5. 5. Pinhole camera terminology Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 5
6. 6. Normalized CameraBy similar triangles: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 6
7. 7. Focal length parametersComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 7
8. 8. Focal length parametersCan model both • the effect of the distance to the focal plane • the density of the receptorswith a single focal length parameterIn practice, the receptors may not be square:So use different focal length parameter for x and y dirns Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 8
9. 9. Offset parameters• Current model assumes that pixel (0,0) is where the principal ray strikes the image plane (i.e. the center)• Model offset to center Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 9
10. 10. Skew parameter• Finally, add skew parameter• Accounts for image plane being not exactly perpendicular to the principal ray Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 10
11. 11. Position and orientation of camera• Position w=(u,v,w)T of point in the world is generally not expressed in the frame of reference of the camera.• Transform using 3D transformation or Point in frame of Point in frame of reference of camera reference of world Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 11
12. 12. Complete pinhole camera model• Intrinsic parameters (stored as intrinsic matrix)• Extrinsic parameters Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 12
13. 13. Complete pinhole camera modelFor short:Add noise – uncertainty in localizing feature in image Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 13
14. 14. Radial distortionComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 14
15. 15. Structure• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems – Exterior orientation problem – Camera calibration – 3D reconstruction• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 15
16. 16. Problem 1: Learning extrinsic parameters (exterior orientation)Use maximum likelihood: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 16
17. 17. Problem 2 – Learning intrinsic parameters (calibration)Use maximum likelihood: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 17
18. 18. Calibration• Use 3D target with known 3D points Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 18
19. 19. Problem 3 – Inferring 3D points (triangulation / reconstruction)Use maximum likelihood: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 19
20. 20. Solving the problems• None of these problems can be solved in closed form• Can apply non-linear optimization to find best solution but slow and prone to local minima• Solution – convert to a new representation (homogeoneous coordinates) where we can solve in closed form.• Caution! We are not solving the true problem – finding global minimum of wrong problem. But can use as starting point for non-linear optimization of true problem Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 20
21. 21. Structure• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems – Exterior orientation problem – Camera calibration – 3D reconstruction• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 21
22. 22. Homogeneous coordinatesConvert 2D coordinate to 3DTo convert back Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 22
23. 23. Geometric interpretation ofhomogeneous coordinates Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 23
24. 24. Pinhole camera in homogeneous coordinatesCamera model:In homogeneous coordinates: (linear!) Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 24
25. 25. Pinhole camera in homogeneous coordinatesWriting out these three equationsEliminate to retrieve original equations Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 25
26. 26. Adding in extrinsic parametersOr for short:Or even shorter: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 26
27. 27. Structure• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems – Exterior orientation problem – Camera calibration – 3D reconstruction• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27
28. 28. Problem 1: Learning extrinsic parameters (exterior orientation)Use maximum likelihood: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 28
29. 29. Exterior orientationStart with camera equation in homogeneous coordinatesPre-multiply both sides by inverse of camera calibration matrix Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 29
30. 30. Exterior orientationStart with camera equation in homogeneous coordinatesPre-multiply both sides by inverse of camera calibration matrix Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 30
31. 31. Exterior orientationThe third equation gives us an expression forSubstitute back into first two lines Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 31
32. 32. Exterior orientationLinear equation – two equations per point –form system of equations Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 32
33. 33. Exterior orientationMinimum direction problem of the form, Find minimum of subject to .To solve, compute the SVD and thenset to the last column of . Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 33
34. 34. Exterior orientationNow we extract the values of and from .Problem: the scale is arbitrary and the rows andcolumns of the rotation matrix may not be orthogonal.Solution: compute SVD and thenchoose .Use the ratio between the rotation matrix before andafter to rescaleUse these estimates for start of non-linear optimisation. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 34
35. 35. Structure• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems – Exterior orientation problem – Camera calibration – 3D reconstruction• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 35
36. 36. Problem 2 – Learning intrinsic parameters (calibration)Use maximum likelihood: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 36
37. 37. CalibrationOne approach (not very efficient) is to alternately• Optimize extrinsic parameters for fixed intrinsic• Optimize intrinsic parameters for fixed extrinsic Then use non-linear optimization. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 37
38. 38. Intrinsic parametersMaximum likelihood approachThis is a least squares problem. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 38
39. 39. Intrinsic parametersThe function is linear w.r.t. intrinsicparameters. Can be written in formNow solve least squares problem Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 39
40. 40. Structure• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems – Exterior orientation problem – Camera calibration – 3D reconstruction• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 40
41. 41. Problem 3 – Inferring 3D points (triangulation / reconstruction)Use maximum likelihood: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 41
42. 42. ReconstructionWrite jth pinhole camera in homogeneous coordinates:Pre-multiply with inverse of intrinsic matrix Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 42
43. 43. ReconstructionLast equations givesSubstitute back into first two equationsRe-arranging get two linear equations for [u,v,w]Solve using >1 cameras and then use non-linear optimization Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 43
44. 44. Structure• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems – Exterior orientation problem – Camera calibration – 3D reconstruction• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 44
45. 45. Depth from structured light Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 45
46. 46. Depth from structured light Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 46
47. 47. Depth from structured light Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 47
48. 48. Shape from silhouetteComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 48
49. 49. Shape from silhouetteComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 49
50. 50. Shape from silhouetteComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 50
51. 51. Conclusion• Pinhole camera model is a non-linear function who takes points in 3D world and finds where they map to in image• Parameterized by intrinsic and extrinsic matrices• Difficult to estimate intrinsic/extrinsic/depth because non-linear• Use homogeneous coordinates where we can get closed from solutions (initial solns only) Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 51