1.
C OMPUTER V ISION : E STIMATION OF 2D H OMOGRAPHY IIT Kharagpur Computer Science and Engineering, Indian Institute of Technology Kharagpur.(IIT Kharagpur) Estimation Jan ’10 1 / 23
2.
We consider a set of point correspondences xi ↔ xi between two images. Such point correspondences may be putative correspondences, i.e. they are believed to be correct, but some of them may be wrong also. Our problem is to compute the 3 × 3 matrix H such that Hxi = xiIssued to be examined: The number of measurements xi ↔ xi required. Should it be just 4? Approximate Solutions? How to compute the homography if there are multiple point correspondences given, some of which may be incorrect or mismatches. Which cost functions should be minimized while estimating the homography? (IIT Kharagpur) Estimation Jan ’10 2 / 23
3.
Direct Linear Transformation (DLT)Algorithm xi = Hxi xi × Hxi = 0 The vector cross product is zero xi × Hxi = 0 because the vectors xi has the same direction as Hxi . The j th row of the matrix H is denoted by hjT 1T h xi Hxi = h2T xi 3T h xi (IIT Kharagpur) Estimation Jan ’10 3 / 23
4.
Writing xi = (xi , yi , wi ) the cross product xi × Hxi = 0 may beexplicitly stated as: yi h3T xi − wi h2T xi xi × Hxi = wi h1T xi − xi h3T xi 2 T x − y h1 T x xi h i i iWe have hj T xi = xiT hj yi xi T h3 − wi xi T h2 xi × Hxi = wi xi T h1 − xi xi T h3 xi xi T h2 − yi xi T h1 (IIT Kharagpur) Estimation Jan ’10 4 / 23
5.
0T −wi xiT yi xiT h1 0T w xT T 2 h =0 i i −xi xi T 3 −yi xiT xi xiT 0 h 1 h h1 h2 h3 Ai h =0 h = h2 H = h4 h5 h6 3 h h7 h8 h9 Aiis a 3 × 9 matrix. h is a 9-vector made up of the entries of thematrix H.The equation Ai h = 0 is an equation linear in the unknown h. (IIT Kharagpur) Estimation Jan ’10 5 / 23
6.
Although there are 3 equations, only 2 of them are linearlyindependent. Thus each point correspondence gives 2 equationsin the entries of h. h1 0T −wi xiT yi xiT 2 h = 0 w xT 0T −xi xiT i i 3 hNow Ai is a 2 × 9 matrix in Ai h = 0The equations hold for any homogeneous coordinaterepresentation (xi , yi , wi )T of the point xi . One can choose thehomogeneous coordinate wi = 1 (IIT Kharagpur) Estimation Jan ’10 6 / 23
7.
Solving for H Each point correspondence gives rise to two independent equations in the entries of H. Given a set of 4 such correspondences we obtain a set of equations Ah = 0 where A is a matrix of equation coefﬁcients built from the matrix rows Ai . The 8 × 9 matrix A has a rank 8, and thus has a 1-dimensional null space which provides for the solution for h. The solution h can only be determined up to a scale. (IIT Kharagpur) Estimation Jan ’10 7 / 23
8.
Over-determined solution What if more than 4 point correspondences xi ↔ xi are available? The set of equatoins Ai h = 0 is over-determined. If the measurement of image coordinates is exact, then there is an exact solution to h. If the image measurements are noisy, there will not be any exact solution apart from the h = 0 solution. Instead of looking for an exact solution, we look for an approximate solution, i.e. a vector h that minimizes a suitable cost function. We attempt to minimize the norm ||Ah|| with constraint ||h|| = 1. This is identical to ﬁnding the minimum of the quotient ||Ah||/||h|| The solution is the (unit) eigenvector of A T A with least eigen value. The solution can also be obtained as the unit singular vector corresponding to the smallest singular value of A. (IIT Kharagpur) Estimation Jan ’10 8 / 23
9.
Degenerate conﬁgurations Suppose that of the given point correspondences, 3 of the points x1 , x2 , x3 are collinear.If the correspondences x1 , x2 , x3 If the correspondences x1 , x2 , x3are all on the same line, it would are not all on the same line, thenmean that the homography is not there can be no projectivesufﬁciently constrained, and there transformation taking xi to xi . Inwill exist a family of homographies such a case the solution matrixmapping xi to xi . It would follow which you will get will be a rank 1that the 8 equations used for matrix and hence does notcomputing H are not independent. represent a projective transformation.A situation where a conﬁguration does not determine a unique solutionfor a particular class of transformations is termed as degenerate. (IIT Kharagpur) Estimation Jan ’10 9 / 23
10.
Data Normalization for DLT algorithm The result of the DLT algorithm for computing 2D homographies depends on the coordinate frame in which the points are expressed. The result of the DLT algorithm is not invariant to similarity transformations of the image. For computing a 2D homography, some coordinate systems turn out to be in some way better than others. Data normalization is carried out prior to homography computation. Such a normalization typically consists of translation and scaling of image coordinates.Data normalization is a compulsory step.An algorithm which incorporates an initial data normalization step willbe invariant with respect to arbitrary choices of scale and coordinateorigin. (IIT Kharagpur) Estimation Jan ’10 10 / 23
11.
Data normalization steps: The points are translated so that their centroid is at the origin. The points are then scaled (isotropic scaling) so that the average √ distance from the origin is equal to 2. This means that the “average" point is equal to (1, 1, 1)T Given two images for homography computation, a (similarity) transformation is applied to each of the two images independently. S1 S2 ˜ ˜ xi −→ xi xi −→ xi Let the transform computed for the ﬁrst image be denoted as S1 and that for the second image be denoted as S2 . ˜ Solve the homography H for the point correspondences: ˜ ˜ xi ↔ xi (IIT Kharagpur) Estimation Jan ’10 11 / 23
12.
Data normalization: DenormalizationThe resulting 2D homography has to be corrected so that it is withrespect to the original coordinate system. H = S−1 H S1 2 ˜ (IIT Kharagpur) Estimation Jan ’10 12 / 23
13.
Symmetric Transfer Error d(xi , H−1 xi )2 + d(xi , Hxi )2 iReprojection Error We are given a set of correspondences xi ↔ x i Our objective is to estimate the homography, as well as a “correction" for each correspondence. ˆ We are seeking a homography H and pairs of perfectly matched ˆ ˆ points xi and xi that minimize the total error function ˆ ˆ d(xi , xi )2 + d(xi , xi )2 ˆ ˆˆ subject to xi = Hxi ∀i i (IIT Kharagpur) Estimation Jan ’10 13 / 23
15.
Robust Estimation The set of correspondences {xi ↔ xi } may have errors. The source of error may be the measurement of the point’s position. Such an error can be modeled using a Gaussian distribution. Errors will also arise if the points are mismatched. The mismatched points are outliers to the Gaussian distribution. These outliers can severely disturb the estimated homography. Robust estimation techniques tend to discard the outliers and retain the inliers. Such techniques are robust (tolerant) to outliers. (IIT Kharagpur) Estimation Jan ’10 15 / 23
16.
RANSAC It is a robust estimator (RANdom SAmple Consensus) algorithm. It is able to cope with a large proportion of outliers. (IIT Kharagpur) Estimation Jan ’10 16 / 23
17.
Consider the problem Given a set of 2D data points, ﬁnd the line which minimizes the sum of squared perpendicular distances, subject to the condition that none of the valid points deviates from this line by more than t units. The problem involves two parts: a line ﬁt to the data; and a classiﬁcation of data into inliers and outliers. (IIT Kharagpur) Estimation Jan ’10 17 / 23
18.
RANSAC Basic Idea Two of the points are selected randomly; these points deﬁne a line. The support for this line is measured by the number of points that lie within a distance threshold. This random selection is repeated a number of times and the line with most support is deemed the robust ﬁt. The points within the threshold distance are the inliers (consensus set). The intuition is that if one of the points (of the minimal subset) is an outlier then the line will not gain much support. Scoring a line by its support has the advantage of favouring a better ﬁt. (IIT Kharagpur) Estimation Jan ’10 18 / 23
19.
RANSAC In general We wish to ﬁt a model to data, and the random sample consists of a minimal subset of the data, sufﬁcient to determine the mode. For a planar homography, the minimal subset consists of 4 point correspondences. The model is a 3 × 3 matrix. (IIT Kharagpur) Estimation Jan ’10 19 / 23
20.
Objective:Robust ﬁt of a model to a data set S which contains outliers. (RANSAC)Algorithm: (i) Randomly select a sample of s data points from S and instantiate the model from this subset. (ii) Determine the set of data points Si which are within a distance threshold t of the model. The set Si is the consensus set of the sample and deﬁnes the inliers of S. (iii) If the size of Si (the number of inliers) is greater than some threshold T , re-estimate the the model using all the points in Si and terminate. (iv) If the size of Si is less than T , select a new subset and repeat the above. (v) After N trials the largest consensus set Si is selected, and the model is re-estimated using all the points in the subset Si . (IIT Kharagpur) Estimation Jan ’10 20 / 23
21.
RANSAC for homography computation Identify by some means a set of putative correspondences. Some of these correspondences may be mismatches. The point matches can be computed automatically based on proximity and similarity of their intensity neighbourhood. RANSAC is designed to 1 Estimate the homography 2 Set of inliers consistent with the estimate (the true correspondences), 3 Outliers (mismatches). (IIT Kharagpur) Estimation Jan ’10 21 / 23
22.
Objective:Compute the 2D homography between two images.Algorithm: (i) Repeat the following for a considerably large number N of samples: (a) Select a random sample of 4 correspondences and compute the homography H (b) Calculate the distance d⊥ for each putative correspondence. (c) Compute the number of inliers consistent with H by the number of correspondences for which d⊥ < t Choose the H with the largest number of inliers. In the case of ties, choose the solution that has the lowest standard deviation of inliers. (IIT Kharagpur) Estimation Jan ’10 22 / 23
23.
Compute the 2D homography between two images.Final Steps: (i) Optimal Estimation: Restimate H from all correspondences classiﬁed as inliers. This can be done by minimizing a suitable cost function. (ii) Guided Matching: Further interest point correspondences are now determined using the estimated H to deﬁne a search region about the transferred point position. These steps can be iterated until the point correspondences stabilize. (IIT Kharagpur) Estimation Jan ’10 23 / 23
Be the first to comment