Robert CollinsCSE486, Penn State                         Lecture 16:                     Planar Homographies
Robert CollinsCSE486, Penn State         Motivation:                     Points on Planar Surface                        y...
Robert CollinsCSE486, Penn State                     Review : Forward Projection       World                Camera        ...
Robert Collins            World to Camera TransformationCSE486, Penn State                                   PC PW        ...
Robert CollinsCSE486, Penn State                     Perspective Matrix Equation       X                       (Camera Coo...
Robert CollinsCSE486, Penn State                         Film to Pixel Coords            2D affine transformation from fil...
Robert Collins               Projection of Points on Planar SurfaceCSE486, Penn State   Perspective   projection          ...
Robert CollinsCSE486, Penn State                     Projection of Planar Points
Robert Collins                Projection of Planar Points (cont)CSE486, Penn State                                        ...
Robert Collins                Projection of Planar Points (cont)CSE486, Penn State                                        ...
Robert CollinsCSE486, Penn State                     Special Case : Frontal Plane        What if the planar surface is per...
Robert CollinsCSE486, Penn State                          Frontal Plane       So the homography for a frontal plane simpli...
Robert CollinsCSE486, Penn State                     Convert to Pixel Coords              Internal camera   a11 a12 a13   ...
Robert Collins                     Planar Projection DiagramCSE486, Penn State  Here’s where  transformation  groups get u...
Robert Collins                     General Planar ProjectionCSE486, Penn State                                       H1   ...
Robert Collins                     General Planar ProjectionCSE486, Penn State                                       H1   ...
Robert Collins                     Frontal Plane ProjectionCSE486, Penn State                                       S1    ...
Robert Collins                     Frontal Plane ProjectionCSE486, Penn State                                       S1    ...
Robert Collins                     General Planar ProjectionCSE486, Penn State                           H1       H2      ...
Robert CollinsCSE486, Penn State                     Summary: Planar Projection                                           ...
Robert Collins              Applying Homographies to RemoveCSE486, Penn State                    Perspective Distortion   ...
Robert CollinsCSE486, Penn State                     Homographies for                     Bird’s-eye Views                ...
Robert CollinsCSE486, Penn State                     Homographies for Mosaicing                                           ...
Robert CollinsCSE486, Penn State                       Two Practical Issues              How to estimate the homography gi...
Robert CollinsCSE486, Penn State                     Estimating a Homography          Matrix Form:          Equations:
Robert CollinsCSE486, Penn State                     Degrees of Freedom?      There are 9 numbers h11,…,h33 , so are there...
Robert CollinsCSE486, Penn State                      Enforcing 8 DOF   One approach: Set h33 = 1.   Second approach: Impo...
Robert CollinsCSE486, Penn State                     L.S. using Algebraic Distance         Setting h33 = 1         Multipl...
Robert CollinsCSE486, Penn State                     Algebraic Distance, h33=1 (cont)                             2N x 8  ...
Robert CollinsCSE486, Penn State                     Algebraic Distance, h33=1 (cont)         Linear                  2Nx8...
Robert CollinsCSE486, Penn State                     Caution: Numeric Conditioning   R.Hartley: “In Defense of the Eight P...
Robert CollinsCSE486, Penn State                     Hartley’s PreConditioning  PointSet1          H = T2-1 S2H?Hnorm S1 T...
Robert CollinsCSE486, Penn State                     A More General Approach         What might be wrong with setting h33 ...
Robert CollinsCSE486, Penn State                     Algebraic Distance, ||h||=1         ||h|| = 1         Multiplying thr...
Robert Collins                 Algebraic Distance, ||h||=1 (cont)CSE486, Penn State                          2N x 9       ...
Robert Collins                 Algebraic Distance, ||h||=1 (cont)CSE486, Penn State         Homogeneous                2Nx...
Upcoming SlideShare
Loading in …5
×

Lecture16

679
-1

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
679
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Lecture16

  1. 1. Robert CollinsCSE486, Penn State Lecture 16: Planar Homographies
  2. 2. Robert CollinsCSE486, Penn State Motivation: Points on Planar Surface y x
  3. 3. Robert CollinsCSE486, Penn State Review : Forward Projection World Camera Film Pixel Coords Coords Coords Coords U X x u V Mext Y Mproj Maff y v W Z U X Mint u V Y v W Z U u M V m11 m12 m13 m14 v W m21 m22 m23 m24 m31 m31 m33 m34
  4. 4. Robert Collins World to Camera TransformationCSE486, Penn State PC PW W Y X U R Z V C Rotate to Translate by - C align axes (align origins) P C = R ( PW - C ) = R PW + T
  5. 5. Robert CollinsCSE486, Penn State Perspective Matrix Equation X (Camera Coordinates) x= f Z y= f Y X  Z  x  f 0 0 0    y  =  0 f Y  0 0    Z   z  0    0 1 0    1   p = M int ⋅ PC
  6. 6. Robert CollinsCSE486, Penn State Film to Pixel Coords 2D affine transformation from film coords (x,y) to pixel coordinates (u,v): X  u’ a11 a12 xa  13 f 0 0 0   v’  ya =  0 a21 a22 23 f Y  0 0 w’    Z  0 0 z1    0   0 1 0    1   Maff Mproj u = Mint PC = Maff Mproj PC
  7. 7. Robert Collins Projection of Points on Planar SurfaceCSE486, Penn State Perspective projection y Film coordinates x Point on plane Rotation + Translation
  8. 8. Robert CollinsCSE486, Penn State Projection of Planar Points
  9. 9. Robert Collins Projection of Planar Points (cont)CSE486, Penn State Homography H (planar projective transformation)
  10. 10. Robert Collins Projection of Planar Points (cont)CSE486, Penn State Homography H (planar projective transformation) Punchline: For planar surfaces, 3D to 2D perspective projection reduces to a 2D to 2D transformation. Punchline2: This transformation is INVERTIBLE!
  11. 11. Robert CollinsCSE486, Penn State Special Case : Frontal Plane What if the planar surface is perpendicular to the optic axis (Z axis of camera coord system)? Then world rotation matrix simplies:
  12. 12. Robert CollinsCSE486, Penn State Frontal Plane So the homography for a frontal plane simplies: Similarity Transformation!
  13. 13. Robert CollinsCSE486, Penn State Convert to Pixel Coords Internal camera a11 a12 a13 v pixels parameters a21 a22 a23 0 0 1 u
  14. 14. Robert Collins Planar Projection DiagramCSE486, Penn State Here’s where transformation groups get useful!
  15. 15. Robert Collins General Planar ProjectionCSE486, Penn State H1 H A1
  16. 16. Robert Collins General Planar ProjectionCSE486, Penn State H1 H-1 A1
  17. 17. Robert Collins Frontal Plane ProjectionCSE486, Penn State S1 A A1
  18. 18. Robert Collins Frontal Plane ProjectionCSE486, Penn State S1 A-1 A1
  19. 19. Robert Collins General Planar ProjectionCSE486, Penn State H1 H2 H
  20. 20. Robert CollinsCSE486, Penn State Summary: Planar Projection Pixel coords Internal u params Perspective v projection y Homography x Point on plane Rotation + Translation
  21. 21. Robert Collins Applying Homographies to RemoveCSE486, Penn State Perspective Distortion from Hartley & Zisserman 4 point correspondences suffice for the planar building facade
  22. 22. Robert CollinsCSE486, Penn State Homographies for Bird’s-eye Views from Hartley & Zisserman
  23. 23. Robert CollinsCSE486, Penn State Homographies for Mosaicing from Hartley & Zisserman
  24. 24. Robert CollinsCSE486, Penn State Two Practical Issues How to estimate the homography given four or more point correspondences (will derive L.S. solution now) How to (un)warp image pixel values to produce a new picture (last class)
  25. 25. Robert CollinsCSE486, Penn State Estimating a Homography Matrix Form: Equations:
  26. 26. Robert CollinsCSE486, Penn State Degrees of Freedom? There are 9 numbers h11,…,h33 , so are there 9 DOF? No. Note that we can multiply all hij by nonzero k without changing the equations:
  27. 27. Robert CollinsCSE486, Penn State Enforcing 8 DOF One approach: Set h33 = 1. Second approach: Impose unit vector constraint Subject to the constraint:
  28. 28. Robert CollinsCSE486, Penn State L.S. using Algebraic Distance Setting h33 = 1 Multiplying through by denominator Rearrange
  29. 29. Robert CollinsCSE486, Penn State Algebraic Distance, h33=1 (cont) 2N x 8 8x1 2N x 1 Point 1 Point 2 Point 3 Point 4 additional points
  30. 30. Robert CollinsCSE486, Penn State Algebraic Distance, h33=1 (cont) Linear 2Nx8 8x1 2Nx1 equations A h = b 8x2N 2Nx8 8x1 8x2N 2Nx1 Solve: AT A h = AT b 8x8 8x1 8x1 (AT A) h = (AT b) -1 h = (AT A) (AT b) Matlab: h = Ab
  31. 31. Robert CollinsCSE486, Penn State Caution: Numeric Conditioning R.Hartley: “In Defense of the Eight Point Algorithm” Observation: Linear estimation of projective transformation parameters from point correspondences often suffer from poor “conditioning” of the matrices involves. This means the solution is sensitive to noise in the points (even if there are no outliers). To get better answers, precondition the matrices by performing a normalization of each point set by: • translating center of mass to the origin • scaling so that average distance of points from origin is sqrt(2). • do this normalization to each point set independently
  32. 32. Robert CollinsCSE486, Penn State Hartley’s PreConditioning PointSet1 H = T2-1 S2H?Hnorm S1 T1 -1 PointSet2 Translate T1 center of mass T2 to origin Scale so average S1 point dist is sqrt(2) S2 Estimate Hnorm
  33. 33. Robert CollinsCSE486, Penn State A More General Approach What might be wrong with setting h33 = 1? If h33 actually = 0, we can’t get the right answer.
  34. 34. Robert CollinsCSE486, Penn State Algebraic Distance, ||h||=1 ||h|| = 1 Multiplying through by denominator Rearrange =0 =0
  35. 35. Robert Collins Algebraic Distance, ||h||=1 (cont)CSE486, Penn State 2N x 9 9x1 2N x 1 4 P O I N T S additional points
  36. 36. Robert Collins Algebraic Distance, ||h||=1 (cont)CSE486, Penn State Homogeneous 2Nx9 9x1 2Nx1 equations A h = 0 9x2N 2Nx9 9x1 9x2N 2Nx1 Solve: AT A h = AT 0 9x9 9x1 9x1 (AT A) h = 0 SVD of ATA = U D UT Let h be the column of U (unit eigenvector) associated with the smallest eigenvalue in D. (if only 4 points, that eigenvalue will be 0)
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×