Lecture 6

123 views
86 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
123
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Lecture 6

  1. 1. C OMPUTER V ISION : S INGULAR VALUE D ECOMPOSITION IIT Kharagpur Computer Science and Engineering, Indian Institute of Technology Kharagpur.(IIT Kharagpur) Minimization Jan ’10 1 / 11
  2. 2. Singular Value Decomposition SVD An m × n matrix of rank r maps an r dimensional unit hypersphere in rowspace (A) into an r -dimensional hyper-ellipse in range (A). The rank 2 matrix  √ √   3 3 1    b = Ax   A = √  −3   3    2 1   1  transforms the unit circle on the plane into an ellipse embedded in three-dimensional space. (IIT Kharagpur) Minimization Jan ’10 2 / 11
  3. 3. Two diametrically opposite points on the unit circle are mappedinto the two endpoints of the major axis of the ellipse, and twoother diametrically opposite points on the unit circle are mappedinto the two endpoints of the minor axis of the ellipse.The lines through these two pairs of points on the unit circle arealways orthogonal. This result can be generalized to any m × nmatrix. (IIT Kharagpur) Minimization Jan ’10 3 / 11
  4. 4. SVDIf A is a real m × n matrix then there exist orthogonal matrices U, V U = [u1 , u2 , . . . , um ] ∈ Rm×m V = [v1 , v2 , . . . , vn ] ∈ Rn×nsuch that T U AV = Σ = diag(σ1 , . . . , σp ) ∈ Rm×nwhere p = min(m, n) and σ1 ≥ σ2 ≥ . . . σp ≥ 0. Equivalently A = UΣV T Vectors v are the right singular vectors Vectors u are the left singular vectors Σ has real non-negative diagonal entries called as singular values. (IIT Kharagpur) Minimization Jan ’10 4 / 11
  5. 5. (IIT Kharagpur) Minimization Jan ’10 5 / 11
  6. 6. SVD SVD is one of the most useful matrix decompositions, particularly for numerical computations. Its most common application is in the solution of over-determined systems of equations. Generally the decomposition is carried out in such a way that the diagonal entries of Σ are in descending order. Since matrices U and V are orthogonal, they have the following properties: ||Ux|| = ||x|| for any vector x T U U = In×n In general UU T is not the identitiy unless m = n. (IIT Kharagpur) Minimization Jan ’10 6 / 11
  7. 7. SVD One can also define SVD for matrices with more columns than rows, but generally this will not be of interest to us. It is appropriate to extend A by adding rows of zeros to obtain a square matrix, and then take the SVD. Common implementations of SVD assume that m ≥ n. In this case matrix U has the same dimension m × n as the input, matrix A may be over-written by the output matrix U. Since matrices U and V are orthogonal, they have the following properties: ||Ux|| = ||x|| for any vector x T U U = In×n In general UU T is not the identitiy unless m = n. (IIT Kharagpur) Minimization Jan ’10 7 / 11
  8. 8. Fitting a line Let pi = (xi , yi )T be a set of m ≥ 2 points on a plane. Consider the line ax + by − c = 0 The normal to the line is given by the vector n = (a, b)T . We can assume that ||n|| = a2 + b2 = 1, thus n is a unit vector. The distance of this line to the origin is |c|, and the distance between the line n and the point pi is di = |axi + byi − c| = |piT n − c| The best fit line minimizes the sum of squared distances. min ||d||2 = min ||Pn − c1||2 ||n||=1 ||n||=1 where P = (p1 , . . . , pm )T , d = (d1 , . . . , dm ), 1 is a vector of m ones. (IIT Kharagpur) Minimization Jan ’10 8 / 11
  9. 9. Fitting a line min ||d||2 = min ||Pn − c1||2 ||n||=1 ||n||=1 Minimization w.r.t to c is done by ∂||d||2 =0 gives c = pT n ∂c 1 T where p is the mean p = mP 1 Substituting for c, the minimization problem w.r.t n is: min ||d||2 = min ||Pn − 1pT n||2 = min ||Qn||2 ||n||=1 ||n||=1 ||n||=1 The solution is n = v2 where v2 is the second column of the matrix V . (IIT Kharagpur) Minimization Jan ’10 9 / 11
  10. 10. function [ l, residue ] = linefit(P)[ m n ] = size(P);if n = 2, error(’matrix P must be m x 2’), endif m < 2, error(’Need at least two points’), endone = ones(m, 1);p = (P’ * one) / m;Q = P - one * p’;[U D V] = svd(Q);n = V(:, 2);l = [n ; p’ * n];residue = D(2, 2); (IIT Kharagpur) Minimization Jan ’10 10 / 11
  11. 11. Singular Values and Eigen Values (IIT Kharagpur) Minimization Jan ’10 11 / 11

×