14 cv mil_the_pinhole_camera

529 views
415 views

Published on

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
529
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
20
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

14 cv mil_the_pinhole_camera

  1. 1. Computer vision: models, learning and inference Chapter 14 The pinhole camera Please send errata to s.prince@cs.ucl.ac.uk
  2. 2. Structure• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems – Exterior orientation problem – Camera calibration – 3D reconstruction• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 2
  3. 3. MotivationSparse stereo reconstructionCompute the depth at a setof sparse matching points Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 3
  4. 4. Pinhole cameraReal camera image Instead model impossible but more is inverted convenient virtual image Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 4
  5. 5. Pinhole camera terminology Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 5
  6. 6. Normalized CameraBy similar triangles: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 6
  7. 7. Focal length parametersComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 7
  8. 8. Focal length parametersCan model both • the effect of the distance to the focal plane • the density of the receptorswith a single focal length parameterIn practice, the receptors may not be square:So use different focal length parameter for x and y dirns Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 8
  9. 9. Offset parameters• Current model assumes that pixel (0,0) is where the principal ray strikes the image plane (i.e. the center)• Model offset to center Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 9
  10. 10. Skew parameter• Finally, add skew parameter• Accounts for image plane being not exactly perpendicular to the principal ray Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 10
  11. 11. Position and orientation of camera• Position w=(u,v,w)T of point in the world is generally not expressed in the frame of reference of the camera.• Transform using 3D transformation or Point in frame of Point in frame of reference of camera reference of world Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 11
  12. 12. Complete pinhole camera model• Intrinsic parameters (stored as intrinsic matrix)• Extrinsic parameters Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 12
  13. 13. Complete pinhole camera modelFor short:Add noise – uncertainty in localizing feature in image Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 13
  14. 14. Radial distortionComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 14
  15. 15. Structure• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems – Exterior orientation problem – Camera calibration – 3D reconstruction• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 15
  16. 16. Problem 1: Learning extrinsic parameters (exterior orientation)Use maximum likelihood: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 16
  17. 17. Problem 2 – Learning intrinsic parameters (calibration)Use maximum likelihood: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 17
  18. 18. Calibration• Use 3D target with known 3D points Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 18
  19. 19. Problem 3 – Inferring 3D points (triangulation / reconstruction)Use maximum likelihood: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 19
  20. 20. Solving the problems• None of these problems can be solved in closed form• Can apply non-linear optimization to find best solution but slow and prone to local minima• Solution – convert to a new representation (homogeoneous coordinates) where we can solve in closed form.• Caution! We are not solving the true problem – finding global minimum of wrong problem. But can use as starting point for non-linear optimization of true problem Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 20
  21. 21. Structure• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems – Exterior orientation problem – Camera calibration – 3D reconstruction• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 21
  22. 22. Homogeneous coordinatesConvert 2D coordinate to 3DTo convert back Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 22
  23. 23. Geometric interpretation ofhomogeneous coordinates Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 23
  24. 24. Pinhole camera in homogeneous coordinatesCamera model:In homogeneous coordinates: (linear!) Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 24
  25. 25. Pinhole camera in homogeneous coordinatesWriting out these three equationsEliminate to retrieve original equations Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 25
  26. 26. Adding in extrinsic parametersOr for short:Or even shorter: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 26
  27. 27. Structure• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems – Exterior orientation problem – Camera calibration – 3D reconstruction• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27
  28. 28. Problem 1: Learning extrinsic parameters (exterior orientation)Use maximum likelihood: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 28
  29. 29. Exterior orientationStart with camera equation in homogeneous coordinatesPre-multiply both sides by inverse of camera calibration matrix Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 29
  30. 30. Exterior orientationStart with camera equation in homogeneous coordinatesPre-multiply both sides by inverse of camera calibration matrix Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 30
  31. 31. Exterior orientationThe third equation gives us an expression forSubstitute back into first two lines Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 31
  32. 32. Exterior orientationLinear equation – two equations per point –form system of equations Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 32
  33. 33. Exterior orientationMinimum direction problem of the form, Find minimum of subject to .To solve, compute the SVD and thenset to the last column of . Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 33
  34. 34. Exterior orientationNow we extract the values of and from .Problem: the scale is arbitrary and the rows andcolumns of the rotation matrix may not be orthogonal.Solution: compute SVD and thenchoose .Use the ratio between the rotation matrix before andafter to rescaleUse these estimates for start of non-linear optimisation. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 34
  35. 35. Structure• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems – Exterior orientation problem – Camera calibration – 3D reconstruction• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 35
  36. 36. Problem 2 – Learning intrinsic parameters (calibration)Use maximum likelihood: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 36
  37. 37. CalibrationOne approach (not very efficient) is to alternately• Optimize extrinsic parameters for fixed intrinsic• Optimize intrinsic parameters for fixed extrinsic Then use non-linear optimization. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 37
  38. 38. Intrinsic parametersMaximum likelihood approachThis is a least squares problem. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 38
  39. 39. Intrinsic parametersThe function is linear w.r.t. intrinsicparameters. Can be written in formNow solve least squares problem Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 39
  40. 40. Structure• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems – Exterior orientation problem – Camera calibration – 3D reconstruction• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 40
  41. 41. Problem 3 – Inferring 3D points (triangulation / reconstruction)Use maximum likelihood: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 41
  42. 42. ReconstructionWrite jth pinhole camera in homogeneous coordinates:Pre-multiply with inverse of intrinsic matrix Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 42
  43. 43. ReconstructionLast equations givesSubstitute back into first two equationsRe-arranging get two linear equations for [u,v,w]Solve using >1 cameras and then use non-linear optimization Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 43
  44. 44. Structure• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems – Exterior orientation problem – Camera calibration – 3D reconstruction• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 44
  45. 45. Depth from structured light Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 45
  46. 46. Depth from structured light Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 46
  47. 47. Depth from structured light Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 47
  48. 48. Shape from silhouetteComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 48
  49. 49. Shape from silhouetteComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 49
  50. 50. Shape from silhouetteComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 50
  51. 51. Conclusion• Pinhole camera model is a non-linear function who takes points in 3D world and finds where they map to in image• Parameterized by intrinsic and extrinsic matrices• Difficult to estimate intrinsic/extrinsic/depth because non-linear• Use homogeneous coordinates where we can get closed from solutions (initial solns only) Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 51

×