Computer vision: models, learning and inference         Chapter 15  Models for transformations   Please send errata to s.p...
Structure• Transformation models• Learning and inference in transformation models• Three problems  – Exterior orientation ...
Transformation models• Consider viewing a planar scene• There is now a 1 to 1 mapping between points  on the plane and poi...
Motivation: augmented reality tracking       Computer vision: models, learning and inference. ©2011 Simon J.D. Prince   4
Euclidean Transformation•   Consider viewing a fronto-parallel plane at a known    distance D.•   In homogeneous coordinat...
Euclidean transformation•   Simplifying•   Rearranging the last equation           Computer vision: models, learning and i...
Euclidean transformation•   Simplifying•   Rearranging the last equation           Computer vision: models, learning and i...
Euclidean transformation•   Pre-multiplying by inverse of (modified) intrinsic matrix           Computer vision: models, l...
Euclidean transformation                                                   Homogeneous:                                   ...
Similarity Transformation•   Consider viewing fronto-parallel plane at unknown    distance D•   By same logic as before we...
Similarity TransformationSimplifying:Multiply each equation by                                               :          Co...
Similarity TransformationSimplifying:Incorporate the constants by defining:          Computer vision: models, learning and...
Similarity Transformation                                          Homogeneous:                                          C...
Affine Transformation                                                 Homogeneous:                                        ...
Affine TransformAffine transform describes mapping well                  When variation in depth is comparable to  when th...
Projective transformation /         collinearity / homographyStart with basic projection equation:Combining these two matr...
Homography                                             Homogeneous:                                              Cartesian...
Adding noise• In the real world, the measured image positions are  uncertain.• We model this with a normal distribution• e...
Structure• Transformation models• Learning and inference in transformation models• Three problems  – Exterior orientation ...
Learning and inference problems• Learning – take points on plane and their  projections into the image and learn  transfor...
Learning and inference problems     Computer vision: models, learning and inference. ©2011 Simon J.D. Prince   21
Learning transformation models• Maximum likelihood approach• Becomes a least squares problem        Computer vision: model...
Learning Euclidean parametersSolve for transformation:Remaining problem:            Computer vision: models, learning and ...
Learning Euclidean parametersHas the general form:•    This is an orthogonal Procrustes problem. To solve:    • Compute SV...
Learning similarity parameters•   Solve for rotation matrix as before•   Solve for translation and scaling factor         ...
Learning affine parameters•   Affine transform is linear•   Solve using least-squares solution            Computer vision:...
Learning homography parameters•    Homography is not linear – cannot be solved in closed     form. Convert to other homoge...
Learning homography parameters•   Write out these equations in full•   There are only 2 independent equations here – use a...
Learning homography parameters•   These equations have the form                                             , which we nee...
Inference problems•   Given point x* in image, find position w* on object           Computer vision: models, learning and ...
InferenceIn the absence of noise, we have the relation                                       ,or in homogeneous coordinate...
Structure• Transformation models• Learning and inference in transformation models• Three problems  – Exterior orientation ...
Problem 1: exterior orientation     Computer vision: models, learning and inference. ©2011 Simon J.D. Prince   33
Problem 1: exterior orientation•   Writing out the camera equations in full•   Estimate the homography from matched points...
Problem 1: exterior orientation•   To estimate the first two columns of rotation matrix, we    compute this singular value...
Problem 1: exterior orientation•   Find the last column using the cross product of first two    columns•   Make sure the d...
Structure• Transformation models• Learning and inference in transformation models• Three problems  – Exterior orientation ...
Problem 2: calibrationComputer vision: models, learning and inference. ©2011 Simon J.D. Prince   38
StructureComputer vision: models, learning and inference. ©2011 Simon J.D. Prince   39
CalibrationOne approach (not very efficient) is to alternately•   Optimize extrinsic parameters for fixed intrinsic•   Opt...
Structure• Transformation models• Learning and inference in transformation models• Three problems  – Exterior orientation ...
Problem 3 - reconstructionTransformation between plane and image:Point in frame of reference of plane:Point in frame of re...
Structure• Transformation models• Learning and inference in transformation models• Three problems  – Exterior orientation ...
Transformations between images•    So far we have considered     transformations between the     image and a plane in the ...
Properties of the homographyHomography is a linear transformation of a rayEquivalently, leave rays and linearly transform ...
Camera under pure rotationSpecial case is camera under pure rotation.Homography can be showed to be         Computer visio...
Structure• Transformation models• Learning and inference in transformation models• Three problems  – Exterior orientation ...
Robust estimation                                                   Least squares criterion is not                        ...
RANSACComputer vision: models, learning and inference. ©2011 Simon J.D. Prince   49
RANSACComputer vision: models, learning and inference. ©2011 Simon J.D. Prince   50
Fitting a homography with RANSAC  Original images                      Initial matches                        Inliers from...
Piecewise planarityMany scenes are not planar, but are nonetheless piecewise planar        Can we match all of the planes ...
Approach 1 – Sequential RANSAC   Problems: greedy algorithm and no need to be spatially coherent        Computer vision: m...
Approach 2 – PEaRL    (propose, estimate and re-learn)•   Associate label l which indicates which plane we are in•   Relat...
Approach 2 – PEaRLComputer vision: models, learning and inference. ©2011 Simon J.D. Prince   55
Structure• Transformation models• Learning and inference in transformation models• Three problems  – Exterior orientation ...
Augmented reality tracking  Computer vision: models, learning and inference. ©2011 Simon J.D. Prince   57
Fast matching of keypoints  Computer vision: models, learning and inference. ©2011 Simon J.D. Prince   58
Visual panoramasComputer vision: models, learning and inference. ©2011 Simon J.D. Prince   59
Conclusions• Mapping between plane in world and camera is one-to-one• Takes various forms, but most general is the homogra...
Upcoming SlideShare
Loading in...5
×

15 cv mil_models_for_transformations

255
-1

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
255
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

15 cv mil_models_for_transformations

  1. 1. Computer vision: models, learning and inference Chapter 15 Models for transformations Please send errata to s.prince@cs.ucl.ac.uk
  2. 2. Structure• Transformation models• Learning and inference in transformation models• Three problems – Exterior orientation – Calibration – Reconstruction• Properties of the homography• Robust estimation of transformations• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 2
  3. 3. Transformation models• Consider viewing a planar scene• There is now a 1 to 1 mapping between points on the plane and points in the image• We will investigate models for this 1 to 1 mapping – Euclidean – Similarity – Affine transform – Homography Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 3
  4. 4. Motivation: augmented reality tracking Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 4
  5. 5. Euclidean Transformation• Consider viewing a fronto-parallel plane at a known distance D.• In homogeneous coordinates the imaging equations are: 3D rotation matrix Plane at known Point is on plane becomes 2D (in plane) distance D (w=0) Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 5
  6. 6. Euclidean transformation• Simplifying• Rearranging the last equation Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 6
  7. 7. Euclidean transformation• Simplifying• Rearranging the last equation Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 7
  8. 8. Euclidean transformation• Pre-multiplying by inverse of (modified) intrinsic matrix Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 8
  9. 9. Euclidean transformation Homogeneous: Cartesian:or for short: For short: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 9
  10. 10. Similarity Transformation• Consider viewing fronto-parallel plane at unknown distance D• By same logic as before we have• Premultiplying by inverse of intrinsic matrix Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 10
  11. 11. Similarity TransformationSimplifying:Multiply each equation by : Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 11
  12. 12. Similarity TransformationSimplifying:Incorporate the constants by defining: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 12
  13. 13. Similarity Transformation Homogeneous: Cartesian: For short: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 13
  14. 14. Affine Transformation Homogeneous: Cartesian: For short:Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 14
  15. 15. Affine TransformAffine transform describes mapping well When variation in depth is comparable to when the depth variation within the distance to object then the affineplanar object is small and the camera is transformation is not a good model. Here far away we need the homography Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 15
  16. 16. Projective transformation / collinearity / homographyStart with basic projection equation:Combining these two matrices we get: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 16
  17. 17. Homography Homogeneous: Cartesian: For short:Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 17
  18. 18. Adding noise• In the real world, the measured image positions are uncertain.• We model this with a normal distribution• e.g. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 18
  19. 19. Structure• Transformation models• Learning and inference in transformation models• Three problems – Exterior orientation – Calibration – Reconstruction• Properties of the homography• Robust estimation of transformations• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 19
  20. 20. Learning and inference problems• Learning – take points on plane and their projections into the image and learn transformation parameters• Inference – take the projection of a point in the image and establish point on plane Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 20
  21. 21. Learning and inference problems Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 21
  22. 22. Learning transformation models• Maximum likelihood approach• Becomes a least squares problem Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 22
  23. 23. Learning Euclidean parametersSolve for transformation:Remaining problem: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 23
  24. 24. Learning Euclidean parametersHas the general form:• This is an orthogonal Procrustes problem. To solve: • Compute SVD • And then set Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 24
  25. 25. Learning similarity parameters• Solve for rotation matrix as before• Solve for translation and scaling factor Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 25
  26. 26. Learning affine parameters• Affine transform is linear• Solve using least-squares solution Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 26
  27. 27. Learning homography parameters• Homography is not linear – cannot be solved in closed form. Convert to other homogeneous coordinates• Both sides are 3x1 vectors; should be parallel and so cross product will be zero Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27
  28. 28. Learning homography parameters• Write out these equations in full• There are only 2 independent equations here – use a minimum of four points to build up a set of equations Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 28
  29. 29. Learning homography parameters• These equations have the form , which we need to solve with the constraint• This is a minimum direction problem – Compute SVD – Take last column of• Then use non-linear optimization Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 29
  30. 30. Inference problems• Given point x* in image, find position w* on object Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 30
  31. 31. InferenceIn the absence of noise, we have the relation ,or in homogeneous coordinatesTo solve for the points, we simply invert this relation Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 31
  32. 32. Structure• Transformation models• Learning and inference in transformation models• Three problems – Exterior orientation – Calibration – Reconstruction• Properties of the homography• Robust estimation of transformations• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 32
  33. 33. Problem 1: exterior orientation Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 33
  34. 34. Problem 1: exterior orientation• Writing out the camera equations in full• Estimate the homography from matched points• Factor out the intrinsic parameters Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 34
  35. 35. Problem 1: exterior orientation• To estimate the first two columns of rotation matrix, we compute this singular value decomposition• Then we set Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 35
  36. 36. Problem 1: exterior orientation• Find the last column using the cross product of first two columns• Make sure the determinant is 1 – if it is -1 then multiply last column by -1.• Find translation scaling factor between old and new values• Finally, set Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 36
  37. 37. Structure• Transformation models• Learning and inference in transformation models• Three problems – Exterior orientation – Calibration – Reconstruction• Properties of the homography• Robust estimation of transformations• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 37
  38. 38. Problem 2: calibrationComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 38
  39. 39. StructureComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 39
  40. 40. CalibrationOne approach (not very efficient) is to alternately• Optimize extrinsic parameters for fixed intrinsic• Optimize intrinsic parameters for fixed extrinsic Then use non-linear optimization. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 40
  41. 41. Structure• Transformation models• Learning and inference in transformation models• Three problems – Exterior orientation – Calibration – Reconstruction• Properties of the homography• Robust estimation of transformations• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 41
  42. 42. Problem 3 - reconstructionTransformation between plane and image:Point in frame of reference of plane:Point in frame of reference of camera Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 42
  43. 43. Structure• Transformation models• Learning and inference in transformation models• Three problems – Exterior orientation – Calibration – Reconstruction• Properties of the homography• Robust estimation of transformations• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 43
  44. 44. Transformations between images• So far we have considered transformations between the image and a plane in the world• Now consider two cameras viewing the same plane• There is a homography between camera 1 and the plane and a second homography between camera 2 and the plane• It follows that the relation between the two images is also a homography Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 44
  45. 45. Properties of the homographyHomography is a linear transformation of a rayEquivalently, leave rays and linearly transform image plane – all images formed byall planes that cut the same ray bundle are related by homographies. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 45
  46. 46. Camera under pure rotationSpecial case is camera under pure rotation.Homography can be showed to be Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 46
  47. 47. Structure• Transformation models• Learning and inference in transformation models• Three problems – Exterior orientation – Calibration – Reconstruction• Properties of the homography• Robust estimation of transformations• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 47
  48. 48. Robust estimation Least squares criterion is not robust to outliers For example, the two outliers here cause the fitted line to be quite wrong. One approach to fitting under these circumstances is to use RANSAC – “Random sampling by consensus”Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 48
  49. 49. RANSACComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 49
  50. 50. RANSACComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 50
  51. 51. Fitting a homography with RANSAC Original images Initial matches Inliers from RANSAC Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 51
  52. 52. Piecewise planarityMany scenes are not planar, but are nonetheless piecewise planar Can we match all of the planes to one another? Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 52
  53. 53. Approach 1 – Sequential RANSAC Problems: greedy algorithm and no need to be spatially coherent Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 53
  54. 54. Approach 2 – PEaRL (propose, estimate and re-learn)• Associate label l which indicates which plane we are in• Relation between points xi in image1 and yi in image 2• Prior on labels is a Markov random field that encourages nearby labels to be similar• Model solved with variation of alpha expansion algorithm Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 54
  55. 55. Approach 2 – PEaRLComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 55
  56. 56. Structure• Transformation models• Learning and inference in transformation models• Three problems – Exterior orientation – Calibration – Reconstruction• Properties of the homography• Robust estimation of transformations• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 56
  57. 57. Augmented reality tracking Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 57
  58. 58. Fast matching of keypoints Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 58
  59. 59. Visual panoramasComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 59
  60. 60. Conclusions• Mapping between plane in world and camera is one-to-one• Takes various forms, but most general is the homography• Revisited exterior orientation, calibration, reconstruction problems from planes• Use robust methods to estimate in the presence of outliers Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 60
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×