2.
What is Camera Calibration?• Primarily, finding the quantities internal to the camera that affect the imaging process § Position of image center in the image • It is typically not at (width/2, height/2) of image § Focal length § Different scaling factors for row pixels and column pixels § Skew factor § Lens distortion (pin-cushion effect)
3.
Motivation • Good calibration is important when we need to § Reconstruct a world model: Virtual L.A. project § Interact with the world • Robot, hand-eye coordination Image plane Evaluation of position of a square for 2 focal lengths (red and blue projection geometry)We see a square ofknown size
4.
Scaling of Rows and Columns in Image • Camera pixels are not necessarily square • Camera output may be analog (NTSC) • Image may be obtained by digitizing card § A/D converter samples NTSC signal Monitor Camera NTSC signal Digitizing displayCCD/CMOS
5.
Compound Lens Imaging• Inexpensive single lens systems distort image at its periphery• Compound lenses may be used to reduce chromatic effects and pin-cushion effects Camera Image plane Nodal Point Principal planes f Image plane for equivalent pinhole camera is f C not camera image plane Center of Projection
6.
Variety of Techniques• VERY large literature on the subject• Work of Roger Tsai influential• Linear algebra method described here § Can be used as initialization for iterative non linear methods.• Some interesting methods use vanishing points
8.
Calibration Procedure• Calibration target: 2 planes at right angle with checkerboard patterns (Tsai grid) § We know positions of pattern corners only with respect to a coordinate system of the target § We position camera in front of target and find images of corners § We obtain equations that describe imaging and contain internal parameters of camera • As a side benefit, we find position and orientation of camera with respect to target (camera pose)
9.
Image Processing of Image of Target• Canny edge detection• Straight line fitting to detected linked edges• Intersecting the lines to obtain the image corners• Matching image corners and 3D target checkerboard corners § By counting if whole target is visible in image• We get pairs (image point)--(world point) ( xi , yi ) → ( X i , Yi , Z i )
10.
Central Projection If world and image points are represented by homogeneous vectors, central projection is a linear transformation: xsxi = f u f 0 xs 0 0 zs v = 0 ys xi = u / w, yi = v / w f 0 0 zs ys w 0 0 1 0 1yi = f Image plane Scene point zs y Image point (xs , ys , zs ) center of (xi , yi , f) x projection z C f
11.
Transformation From Lengths to Pixels Transformation uses: y Image point ypix (xi , yi , f) • image center (x0, y0) y0 Image center x • scaling factors kx and ky x0 xpix xs xs + zs x0 xi = f x pix = k x xi + x0 = f k x zs zs ys y s + z s y0 yi = f y pix = k y yi + y0 = f k y zs zs xs u α x 0 x0 0 α x = f kx x pix = u / w v = 0 α y0 ys 0 with then y zs α y = f ky y pix = v / ww 0 0 1 0 1
12.
Internal Camera Parameters xs u α x s x0 0 α x = f kx x pix = u / w v = 0 α y y0 0 ys with zs α y = − f ky y pix = v / ww 0 0 1 0 1 α x s x0 0 α x s x0 1 0 0 0 0 α y0 0 = 0 α y y0 0 1 0 0 = K [I 3 | 03 ] y 0 0 1 0 0 0 1 0 0 1 0 • αx and αy “focal lengths” in pixels • x0 and y0 coordinates of image center in pixels •Added parameter s is skew parameter • K is called calibration matrix. Five degrees of freedom. •K is a 3x3 upper triangular matrix
13.
From Camera Coordinates y to World CoordinatesC x Image planeImage point Z z (xi , yi , f) Y (xs , ys , zs ) M O (Xs , Ys , Z s ) X
14.
From Camera Coordinates to World Coordinates 2 x X i k z M j (Xs , Ys , Z s ) in world cordinate system C I (xs , ys , zs ) in image cordinate systemCO = T translation vector J O K Z CM = CO + OM xS i + y S j + zS k = Tx i + Ty j + Tz k + X S I + YS J + Z S K xS = Tx + X S I.i + YS J.i + Z S K.i xS Tx I.i J.iK.i X S y = T + I.j J.j K.j YS S y z S Tz I.k J.k K.k Z S
15.
Homogeneous Coordinates xS Tx I.i J.i K.i X S y = T + I.j J.j K.j Y S y S z S Tz I.k J.k K.k Z S xS I.i J.i Tx X S K.i xS X S y I.j y T YS S = J.j Ty YS K.j S = R z S I.k J.k K.k Tz Z S z S 0 T 3 Z 1 S 1 0 0 0 1 1 1 1
16.
From Camera Coordinates to World Coordinates 3 x Xi k z Mj (Xs , Ys , Z s ) C I (xs , ys , zs ) J O K ZCM = OM - OCxS i + yS j + z S k = ( X S - X C ) I + (YS - YC ) J + ( Z S - Z C ) KxS = ( X S - X C ) I.i + (YS - YC ) J.i + ( Z S - Z C ) K.i ~ ~ x cam = R (X - C) (T = -R C) ~ C is vector OC expressed in world coordinate system
17.
Homogeneous Coordinates 2 ~• Here we use - R C instead of T xS XS y ~ S = R - R C YS z S 0 T 1 ZS 3 1 1
18.
Linear Transformation from World Coordinates to Pixels • Combine camera projection and coordinate transformation matrices into a single matrix P xs u y v = K [I | 0 3 ] s 3 zs XS w u ~ 1 v = K [I | 0 ] R ⇒ - R C YS 3 T 1 Z S 3 xS XS w 0 3 y ~ 1 S = R - R C YS XS z S 0 T 1 ZS u Y ⇒ v = P S 3 x = PX1 1 ZS w 1
19.
Properties of Matrix P• Further simplification of P: R ~ - R Cx = K [I 3 | 0 3 ] T X 0 3 1 ~ [I 3 R | 0 3 ] T - R C [ ~ ] [ ~ = R - R C = R I3 | -C ] 0 3 1 [ ~x = K R I3 | -C X ] [ ~ P = K R I3 | - C ]• P has 11 degrees of freedom: ~ • 5 from triangular calibration matrix K, 3 from R and 3 from C• P is a fairly general 3 x 4 matrix •left 3x3 submatrix KR is non-singular
20.
Calibration• 1. Estimate matrix P using scene points and their images• 2. Estimate the interior parameters and the exterior parameters [ ~ P = K R I3 | - C ] § Left 3x3 submatrix of P is product of upper- triangular matrix and orthogonal matrix
21.
Finding Camera Translation• Find homogeneous coordinates of C in the scene• C is the null vector of matrix P § P C = 0: X c 0 1 0 0 X c P = K R I3[ ~ | -C ] 0 1 0 Yc Yc = 0 Z c 0 0 0 1 Zc 1 0• Find null vector C of P using SVD § C is the unit singular vector of P corresponding to the smallest singular value (the last column of V, where P = U D VT is the SVD of P)
22.
Finding Camera Orientation and Internal Parameters• Left 3x3 submatrix M of P is of form M=K R § K is an upper triangular matrix § R is an orthogonal matrix• Any non-singular square matrix M can be decomposed into the product of an upper- triangular matrix K and an orthogonal matrix R using the RQ factorization § Similar to QR factorization but order of 2 matrices is reversed
23.
RQ Factorization of M 1 0 0 c 0 s c − s 0R x = 0 c − s , R y = 0 1 0 , R z = s c 0 0 s c − s 0 c 0 0 1 m33 m32• Compute c = − , s= 2 ( m32 + m33 ) 2 2 1/ 2 (m32 + m33 )1/ 2 2• Multiply M by Rx. The resulting term at (3,2) is zero because of the values selected for c and s• Multiply the resulting matrix by Ry, after selecting c’ and s’ so that the resulting term at position (3,1) is set to zero• Multiply the resulting matrix by Rz, after selecting c’’ and s’’ so that the resulting term at position (2,1) is set to zero M RxR yR z = K ⇒ M = K R TRTRT = K R z y x
24.
Computing Matrix P• Use corresponding image and scene points § 3D points Xi in world coordinate system § Images xi of Xi in image• Write xi = P Xi for all i• Similar problem to finding projectivity matrix H (i.e. homography) in homework
25.
Improved Computation of P • xi = P Xi involves homogeneous coordinates, thus xi and P Xi just have to be proportional: x i × P Xi = 0 • Let p1T, p2T, p3T be the 3 row vectors of P p 1 X i T vi p T Xi − wi p T Xi 3 2 T P X i = p 2 X i xi × P X i = wi p1 Xi − u i p 3 Xi T T p T X i ui p T X i − vi p1 X i T 3 2 0T − wi X T v i X T p 1 p1 4 i i T ⇒ w i X Ti 0T4 − ui X i p 2 = 0 p2 is a 12 × 1 vector − v i X T u i X T 0T p 3 p3 i i 4
26.
Improved Computation of P, cont’d• Third row can be obtained from sum of u’i times first row - v’i times second row 0T 4 − wi X T i v i X T p 1 i wi X T i 0T4 − u i X T p 2 = 0 i − v i X T u i X T 0T p 3 i i 4 • So we get 2 independent equations in 11 unknowns (ignoring scale)• With 6 point correspondences, we get enough equations to compute matrix P Ap=0
27.
Solving A p = 0• Linear system A p = 0• When possible, have at least 5 times as many equations as unknowns (28 points)• Minimize || A p || with the constraint || p || = 1 § P is the unit singular vector of A corresponding to the smallest singular value (the last column of V, where A = U D V T is the SVD of A)• Called Direct Linear Transformation (DLT)
28.
Improving P Solution with Nonlinear Minimization• Find p using DLT• Use as initialization for nonlinear minimization of ∑ d ( x i , PX i ) 2 i § Use Levenberg-Marquardt iterative minimization
29.
Radial Distortion• We have assumed that lines are imaged as lines• Not quite true for real lenses § Significant error for cheap optics and for short focal lengths Scene Image Corrected image
30.
Radial Distortion Modeling• In pixel cordinates the correction is written xc − x0 = L(r ) ( x − x0 ) (x, y) yc − y0 = L ( r ) ( y − y0 ) (xc, yc) (x0, y0) with Distorted line r 2 = ( x − x0 ) 2 + ( y − y0 ) 2 And ideal line: L(r ) = 1 + κ 1 r + κ 2 r 2 + ... (x, y) (x’, y’)• Minimize f (κ 1 ,κ 2 ) = ∑ ( xi − xci ) 2 + ( y i − yci ) 2 i using lines known to be straight (x’,y’) is radial projection of (x,y) on straight line
31.
References• Multiple View Geometry in Computer Vision, R. Hartley and A. Zisserman, Cambridge University Press, 2000, pp. 138-183• Three-Dimensional Computer Vision: A Geometric Approach, O. Faugeras, MIT Press, 1996, pp. 33-68• “A Versatile Camera Calibration Technique for 3D Machine Vision”, R. Y. Tsai, IEEE J. Robotics & Automation, RA-3, No. 4, August 1987, pp. 323-344
A particular slide catching your eye?
Clipping is a handy way to collect important slides you want to go back to later.
Be the first to comment