Camera Calibration
What is Camera Calibration?• Primarily, finding the quantities internal to  the camera that affect the imaging process  § ...
Motivation    • Good calibration is important when we need      to         § Reconstruct a world model: Virtual L.A. proje...
Scaling of Rows and Columns              in Image • Camera pixels are not necessarily square • Camera output may be analog...
Compound Lens Imaging• Inexpensive single lens systems distort image at  its periphery• Compound lenses may be used to red...
Variety of Techniques• VERY large literature on the subject• Work of Roger Tsai influential• Linear algebra method describ...
Camera and Calibration Target
Calibration Procedure• Calibration target: 2 planes at right angle with  checkerboard patterns (Tsai grid)  § We know posi...
Image Processing of           Image of Target• Canny edge detection• Straight line fitting to detected linked edges• Inter...
Central Projection If world and image points are represented by homogeneous vectors, central projection is a linear transf...
Transformation From Lengths to Pixels   Transformation uses:                                              y        Image p...
Internal Camera Parameters                             xs  u   α x     s x0 0               α x = f kx     x pix =...
From Camera Coordinates    y   to World CoordinatesC          x                 Image planeImage point                    ...
From Camera Coordinates                   to World Coordinates 2          x                                               ...
Homogeneous Coordinates xS  Tx   I.i J.i K.i   X S  y  = T  +  I.j J.j K.j   Y  S  y                 ...
From Camera Coordinates                 to World Coordinates 3        x                                           Xi      ...
Homogeneous Coordinates 2                  ~• Here we use - R C instead of T             xS                 XS        ...
Linear Transformation from          World Coordinates to Pixels  • Combine camera projection and coordinate    transformat...
Properties of Matrix P• Further simplification of P:                           R            ~                            ...
Calibration• 1. Estimate matrix P using scene points and      their images• 2. Estimate the interior parameters and the   ...
Finding Camera Translation• Find homogeneous coordinates of C in the  scene• C is the null vector of matrix P  § P C = 0: ...
Finding Camera Orientation and        Internal Parameters• Left 3x3 submatrix M of P is of form M=K R  § K is an upper tri...
RQ Factorization of M      1 0 0             c 0 s        c  − s  0R x = 0 c − s  , R y =  0 1 0  , R z = s   c...
Computing Matrix P• Use corresponding image and scene points  § 3D points Xi in world coordinate system  § Images xi of Xi...
Improved Computation of P      • xi = P Xi involves homogeneous        coordinates, thus xi and P Xi just have to be      ...
Improved Computation of P,            cont’d• Third row can be obtained from sum of u’i  times first row - v’i times secon...
Solving A p = 0• Linear system A p = 0• When possible, have at least 5 times as  many equations as unknowns (28 points)• M...
Improving P Solution with      Nonlinear Minimization• Find p using DLT• Use as initialization for nonlinear  minimization...
Radial Distortion• We have assumed that lines are imaged as  lines• Not quite true for real lenses  § Significant error fo...
Radial Distortion Modeling• In pixel cordinates the correction is written                       xc − x0 = L(r ) ( x − x0 )...
References• Multiple View Geometry in Computer Vision, R.  Hartley and A. Zisserman, Cambridge University  Press, 2000, pp...
Upcoming SlideShare
Loading in...5
×

Lecture9 camera calibration

621

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
621
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
32
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Lecture9 camera calibration

  1. 1. Camera Calibration
  2. 2. What is Camera Calibration?• Primarily, finding the quantities internal to the camera that affect the imaging process § Position of image center in the image • It is typically not at (width/2, height/2) of image § Focal length § Different scaling factors for row pixels and column pixels § Skew factor § Lens distortion (pin-cushion effect)
  3. 3. Motivation • Good calibration is important when we need to § Reconstruct a world model: Virtual L.A. project § Interact with the world • Robot, hand-eye coordination Image plane Evaluation of position of a square for 2 focal lengths (red and blue projection geometry)We see a square ofknown size
  4. 4. Scaling of Rows and Columns in Image • Camera pixels are not necessarily square • Camera output may be analog (NTSC) • Image may be obtained by digitizing card § A/D converter samples NTSC signal Monitor Camera NTSC signal Digitizing displayCCD/CMOS
  5. 5. Compound Lens Imaging• Inexpensive single lens systems distort image at its periphery• Compound lenses may be used to reduce chromatic effects and pin-cushion effects Camera Image plane Nodal Point Principal planes f Image plane for equivalent pinhole camera is f C not camera image plane Center of Projection
  6. 6. Variety of Techniques• VERY large literature on the subject• Work of Roger Tsai influential• Linear algebra method described here § Can be used as initialization for iterative non linear methods.• Some interesting methods use vanishing points
  7. 7. Camera and Calibration Target
  8. 8. Calibration Procedure• Calibration target: 2 planes at right angle with checkerboard patterns (Tsai grid) § We know positions of pattern corners only with respect to a coordinate system of the target § We position camera in front of target and find images of corners § We obtain equations that describe imaging and contain internal parameters of camera • As a side benefit, we find position and orientation of camera with respect to target (camera pose)
  9. 9. Image Processing of Image of Target• Canny edge detection• Straight line fitting to detected linked edges• Intersecting the lines to obtain the image corners• Matching image corners and 3D target checkerboard corners § By counting if whole target is visible in image• We get pairs (image point)--(world point) ( xi , yi ) → ( X i , Yi , Z i )
  10. 10. Central Projection If world and image points are represented by homogeneous vectors, central projection is a linear transformation: xsxi = f u   f 0  xs  0 0   zs v  =  0   ys  xi = u / w, yi = v / w    f 0 0  zs  ys  w  0    0 1 0    1yi = f   Image plane Scene point zs y Image point (xs , ys , zs ) center of (xi , yi , f) x projection z C f
  11. 11. Transformation From Lengths to Pixels Transformation uses: y Image point ypix (xi , yi , f) • image center (x0, y0) y0 Image center x • scaling factors kx and ky x0 xpix xs xs + zs x0 xi = f x pix = k x xi + x0 = f k x zs zs ys y s + z s y0 yi = f y pix = k y yi + y0 = f k y zs zs  xs  u  α x 0 x0 0   α x = f kx x pix = u / w v  =  0 α y0   ys  0   with then y  zs  α y = f ky y pix = v / ww  0   0 1 0    1  
  12. 12. Internal Camera Parameters  xs  u  α x s x0 0    α x = f kx x pix = u / w v  =  0 α y y0 0   ys    with  zs  α y = − f ky y pix = v / ww  0   0 1 0    1   α x s x0 0 α x s x0  1 0 0 0 0 α y0 0 =  0 α y y0  0 1 0 0 = K [I 3 | 03 ]  y     0  0 1 0  0   0 1  0 0 1 0   • αx and αy “focal lengths” in pixels • x0 and y0 coordinates of image center in pixels •Added parameter s is skew parameter • K is called calibration matrix. Five degrees of freedom. •K is a 3x3 upper triangular matrix
  13. 13. From Camera Coordinates y to World CoordinatesC x Image planeImage point Z z (xi , yi , f) Y (xs , ys , zs ) M O (Xs , Ys , Z s ) X
  14. 14. From Camera Coordinates to World Coordinates 2 x X i k z M j (Xs , Ys , Z s ) in world cordinate system C I (xs , ys , zs ) in image cordinate systemCO = T translation vector J O K Z CM = CO + OM xS i + y S j + zS k = Tx i + Ty j + Tz k + X S I + YS J + Z S K xS = Tx + X S I.i + YS J.i + Z S K.i  xS  Tx   I.i J.iK.i   X S   y  = T  +  I.j J.j K.j   YS   S   y     z S  Tz  I.k      J.k K.k   Z S   
  15. 15. Homogeneous Coordinates xS  Tx   I.i J.i K.i   X S  y  = T  +  I.j J.j K.j   Y  S  y   S  z S  Tz  I.k J.k K.k   Z S         xS   I.i J.i Tx   X S  K.i  xS  X S   y   I.j y  T  YS   S =  J.j Ty   YS  K.j    S = R    z S  I.k J.k K.k Tz   Z S   z S  0 T  3  Z  1 S          1  0 0 0 1  1  1  1 
  16. 16. From Camera Coordinates to World Coordinates 3 x Xi k z Mj (Xs , Ys , Z s ) C I (xs , ys , zs ) J O K ZCM = OM - OCxS i + yS j + z S k = ( X S - X C ) I + (YS - YC ) J + ( Z S - Z C ) KxS = ( X S - X C ) I.i + (YS - YC ) J.i + ( Z S - Z C ) K.i ~ ~ x cam = R (X - C) (T = -R C) ~ C is vector OC expressed in world coordinate system
  17. 17. Homogeneous Coordinates 2 ~• Here we use - R C instead of T  xS  XS  y   ~    S = R - R C  YS    z S  0 T 1  ZS      3 1  1 
  18. 18. Linear Transformation from World Coordinates to Pixels • Combine camera projection and coordinate transformation matrices into a single matrix P  xs  u y   v  = K [I | 0 3 ] s    3  zs  XS  w     u  ~   1  v  = K [I | 0 ] R ⇒  - R C  YS  3  T  1  Z S  3 xS  XS   w 0 3    y   ~    1  S = R - R C  YS   XS  z S  0 T 1  ZS  u Y     ⇒  v  = P  S  3   x = PX1  1  ZS  w     1 
  19. 19. Properties of Matrix P• Further simplification of P: R ~ - R Cx = K [I 3 | 0 3 ] T X 0 3 1  ~ [I 3 R | 0 3 ] T - R C  [ ~ ] [ ~ = R - R C = R I3 | -C ] 0 3 1  [ ~x = K R I3 | -C X ] [ ~ P = K R I3 | - C ]• P has 11 degrees of freedom: ~ • 5 from triangular calibration matrix K, 3 from R and 3 from C• P is a fairly general 3 x 4 matrix •left 3x3 submatrix KR is non-singular
  20. 20. Calibration• 1. Estimate matrix P using scene points and their images• 2. Estimate the interior parameters and the exterior parameters [ ~ P = K R I3 | - C ] § Left 3x3 submatrix of P is product of upper- triangular matrix and orthogonal matrix
  21. 21. Finding Camera Translation• Find homogeneous coordinates of C in the scene• C is the null vector of matrix P § P C = 0:  X c  0 1 0 0 X c     P = K R I3[ ~ | -C ] 0 1 0  Yc   Yc  = 0  Z c  0 0 0 1  Zc      1   0• Find null vector C of P using SVD § C is the unit singular vector of P corresponding to the smallest singular value (the last column of V, where P = U D VT is the SVD of P)
  22. 22. Finding Camera Orientation and Internal Parameters• Left 3x3 submatrix M of P is of form M=K R § K is an upper triangular matrix § R is an orthogonal matrix• Any non-singular square matrix M can be decomposed into the product of an upper- triangular matrix K and an orthogonal matrix R using the RQ factorization § Similar to QR factorization but order of 2 matrices is reversed
  23. 23. RQ Factorization of M 1 0 0   c 0 s  c − s 0R x = 0 c − s  , R y =  0 1 0  , R z = s c 0       0 s c    − s 0 c   0  0 1  m33 m32• Compute c = − , s= 2 ( m32 + m33 ) 2 2 1/ 2 (m32 + m33 )1/ 2 2• Multiply M by Rx. The resulting term at (3,2) is zero because of the values selected for c and s• Multiply the resulting matrix by Ry, after selecting c’ and s’ so that the resulting term at position (3,1) is set to zero• Multiply the resulting matrix by Rz, after selecting c’’ and s’’ so that the resulting term at position (2,1) is set to zero M RxR yR z = K ⇒ M = K R TRTRT = K R z y x
  24. 24. Computing Matrix P• Use corresponding image and scene points § 3D points Xi in world coordinate system § Images xi of Xi in image• Write xi = P Xi for all i• Similar problem to finding projectivity matrix H (i.e. homography) in homework
  25. 25. Improved Computation of P • xi = P Xi involves homogeneous coordinates, thus xi and P Xi just have to be proportional: x i × P Xi = 0 • Let p1T, p2T, p3T be the 3 row vectors of P p 1 X i  T  vi p T Xi − wi p T Xi  3 2  T   P X i = p 2 X i  xi × P X i =  wi p1 Xi − u i p 3 Xi  T T p T X i   ui p T X i − vi p1 X i  T  3   2   0T − wi X T v i X T   p 1  p1   4 i i T    ⇒  w i X Ti 0T4 − ui X i  p 2  = 0 p2  is a 12 × 1 vector  − v i X T u i X T 0T  p 3  p3   i i 4    
  26. 26. Improved Computation of P, cont’d• Third row can be obtained from sum of u’i times first row - v’i times second row  0T 4 − wi X T i v i X T   p 1  i     wi X T i 0T4 − u i X T  p 2  = 0 i  − v i X T u i X T 0T  p 3   i i 4  • So we get 2 independent equations in 11 unknowns (ignoring scale)• With 6 point correspondences, we get enough equations to compute matrix P Ap=0
  27. 27. Solving A p = 0• Linear system A p = 0• When possible, have at least 5 times as many equations as unknowns (28 points)• Minimize || A p || with the constraint || p || = 1 § P is the unit singular vector of A corresponding to the smallest singular value (the last column of V, where A = U D V T is the SVD of A)• Called Direct Linear Transformation (DLT)
  28. 28. Improving P Solution with Nonlinear Minimization• Find p using DLT• Use as initialization for nonlinear minimization of ∑ d ( x i , PX i ) 2 i § Use Levenberg-Marquardt iterative minimization
  29. 29. Radial Distortion• We have assumed that lines are imaged as lines• Not quite true for real lenses § Significant error for cheap optics and for short focal lengths Scene Image Corrected image
  30. 30. Radial Distortion Modeling• In pixel cordinates the correction is written xc − x0 = L(r ) ( x − x0 ) (x, y) yc − y0 = L ( r ) ( y − y0 ) (xc, yc) (x0, y0) with Distorted line r 2 = ( x − x0 ) 2 + ( y − y0 ) 2 And ideal line: L(r ) = 1 + κ 1 r + κ 2 r 2 + ... (x, y) (x’, y’)• Minimize f (κ 1 ,κ 2 ) = ∑ ( xi − xci ) 2 + ( y i − yci ) 2 i using lines known to be straight (x’,y’) is radial projection of (x,y) on straight line
  31. 31. References• Multiple View Geometry in Computer Vision, R. Hartley and A. Zisserman, Cambridge University Press, 2000, pp. 138-183• Three-Dimensional Computer Vision: A Geometric Approach, O. Faugeras, MIT Press, 1996, pp. 33-68• “A Versatile Camera Calibration Technique for 3D Machine Vision”, R. Y. Tsai, IEEE J. Robotics & Automation, RA-3, No. 4, August 1987, pp. 323-344
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×