This document discusses various mathematical tools used for monocular model-based 3D tracking of rigid objects, including camera representation using the pinhole camera model and calibration matrix, methods for estimating the external camera parameters matrix like DLT and PnP, and techniques for pose estimation, robust estimation, and Bayesian tracking. It covers topics like camera parameterization using Euler angles, quaternions and exponential maps, and algorithms for minimizing reprojection error like linear least squares and Newton-based methods. It also describes M-estimators and RANSAC for robust estimation in the presence of outliers.

1. Monocular Model-Based 3D Tracking of Rigid Objects: A Survey2008. 12. 04.백운혁Chapter 2. Mathematical Tools

3. AgendaMonocular Model-Based 3D Tracking of Rigid Objects : A SurveyChapter 2. Mathematical Tools2.1 Camera Representation2.2 Camera Pose Parameterization2.3 Estimating the External Parameters Matrix2.4 Least-Squares Minimization Techniques2.5 Robust Estimation2.6 Bayesian Tracking

4. the standard pinhole camera model2.1 Camera Representation

5. 2.1.1 The Perspective Projection ModelImage coordinate systemWorld CoordinatesImage Coordinates(in the image)Projection Matrix

6. 2.1.2 The Camera Calibration Matrixinternal parametersfocal lengthprincipal pointskew parameterthe number of pixels per unit distance in the uthe number of pixels per unit distance in the v

7. 2.1.2 The Camera Calibration Matrixprojectionfocal lengthImage Plane

8. 2.1.2 The Camera Calibration Matrixprojection to imageprincipal point (center of image plane)the number of pixels per unit distance in the uthe number of pixels per unit distance in the v

9. 2.1.2 The Camera Calibration Matrixskewfield of viewreferred as the skew, usuallyimage plane size and field of view are assumed to be fixed,but not fixed focal length

10. 2.1.3 The External Parameters Matrixworld coordinate to camera coordinateThe 3x4 external parametersrotation matrixtranslation vectorin the world coordinate systemin the camera coordinate system

11. 2.1.3 The External Parameters Matrix

12. 2.1.4 Estimating the Camera Calibration Matrixinternal parameters are assumed to be fixedmake use of a calibration pattern of known sizeinside the field of viewcorrespondence between the 3D points and the 2D image points

13. 2.1.5 Handling Lens Distortion(usually ignored)radial distortiontangential distortion

14. can be avoided by locally re-pametrizing the rotation2.2 Camera Pose Parameterization

15. 2.2.1 Euler Anglesα,β,γ to be rotation angles around the Z, Y, and X axis respectively yields

16. one rotation has no effect

17. gimbal lock problem2.2.2 QuaternionsA rotation about the unit vector by an angle a scalar plus a 3-vector2.2.3 Exponential MapA rotation about the unit vector by an angle Let be a 3D vector2.2.3 Exponential MapRodrigues’ formulathe exponential map represents a rotation as a 3-vector that gives its axis and magnitude.is the skew-symmetric matrix

18. 2.2.4 Linearization of Small Rotations

19. estimated camera positions(when the internal parameters are known)2.3 Estimating the External parameters Matrix

20. 2.3.1 How many Correspondences are necessary?n=3 known correspondences produce 4 possible solution (P3P Problem)n>=4 known correspondences produce 2 possible solutionn>=4 known correspondences (points are coplanar) produce unique solutionn>=6 known correspondences produce unique solution

21. 2.3.2 The Direct Linear Transformation (DLT)to estimate the whole matrix P by solving a linear system even when the internal parameters are not knownEach correspondence gives rise to two linearly independent equations

22. 2.3.2 The Direct Linear Transformation (DLT)Stacking all the equation into B yields the linear system :

23. 2.3.2 The Direct Linear Transformation (DLT)is the eigen vector of B corresponding to the smallest eigenvalue of B6 correspondences must be knownfor 3D tracking , using a calibrated camera and estimating only its orientation and position

24. 2.3.3 The Perspective-n-Point (PnP) Problem

25. 2.3.4 Pose estimation from a 3D PlaneThe relation between a 3D plane and its image projection can be represented by a homogeneous 3x3 matrix (homography matrix)Let us consider the plane

26. 2.3.4 Pose estimation from a 3D PlaneThe matrix H can be estimated from four correspondences using a DLT algorithm the translation vector last column is given by the cross-productsince the columns of R must be orthonormal

27. 2.3.5 non-Linear Reprojection Error

28. finding the pose that minimizes a sum of residual errors 2.4 Least-Squares Minimization Techniques

29. 2.4.1 Linear Least-Squaresthe function is linearthe camera pose parameters the unknowns of a set of linear equations in matrix form as can be estimated as pseudo-inverse of A

30. 2.4.2 Newton-Based Minimization Algorithmsthe function is not linearalgorithms start from an initial estimate of the minimum and update it iteratively is chosen to minimize the residual at iteration and estimated by approximating to the first order

31. 2.4.2 Newton-Based Minimization AlgorithmsJacobian matrix the partial derivatives of all these functionsstabilizes the begavior

32. inliers data whose distribution can be explained by some set of model parametersoutliers which are data that do not fit the model the data can be subject to noiseM-estimators good at finding accurate solutions require an initial estimate to converge correctlyRANSAC does not require such an initial estimate does not take into account all the available data lacks precision2.5 Robust Estimation

33. 2.5.1 M-Estimatorsleast-squares estimationthe assumption that the observations are independent and have a Gaussian distributionInstead of minimizingare residual errorsis an M-estimator that reduce the influence of outliers

34. 2.5.1 M-EstimatorsHuber estimatorTukey estimator

35. Huber estimator : linear to reduce the influence of large residual errors

36. Tukey estimator : flat so that large residual errors have no influence at all2.5.1 M-Estimators

37. 2.5.2 RANSAC samples of data pointsare randomly selectedestimate model parametersfind the subset of points (consistent with the estimate)the largest is retained and refined by least-squares minimizationthe model parameters require a minimum ofa set of measurements

38. 2.5.2 RANSAClinear least-square estimation

39. 2.5.2 RANSACrandom sampling

40. 2.5.2 RANSACrandom sampling

41. 2.5.2 RANSACrandom sampling

42. 2.5.2 RANSACrandom sampling

43. estimating the density of successive states in the space of possible camera poses.2.6 Bayesian Tracking

45. Thank you for your attention