Your SlideShare is downloading. ×
  • Like
SVD review
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply
Published

 

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,868
On SlideShare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
52
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. CS 395/495-26: Spring 2004
    • IBMR: Singular Value Decomposition (SVD Review)
    • Jack Tumblin
    • [email_address]
  • 2. Matrix Multiply: A Change-of-Axes
    • Matrix Multiply: Ax = b
      • x and b are column vectors
      • A has m rows, n columns
    x 1 … x n a 11 … a 1n … … a m1 … a mn = b 1 … b m x 1 x 2 b 3 b 2 b 1 Ax=b x b Input (N dim) (2D) Output (M-dim) (3D)
  • 3. Matrix Multiply: A Change-of-Axes
    • Matrix Multiply: Ax = b
      • x and b are column vectors
      • A has m rows, n columns
    • Matrix multiply is just a set of dot-products; it ‘changes coordinates’ for vector x, makes b.
      • Rows of A = A 1 ,A 2 ,A 3 ,... = new coordinate axes
      • Ax = a dot product for each new axis
    x 1 … x n a 11 … a 1n … … a m1 … a mn = b 1 … b m x 1 x 2 A 2 A 1 A 3 b 3 b 2 b 1 Ax=b x b Input (N dim) (2D) Output (M-dim) (3D)
  • 4.
    • Sphere of all unit-length x  ellipsoid of b
    • Output ellipsoid’s axes are always perpendicular (true for any ellipsoid)
    • **THUS** we can make ┴ , unit-length axes…
    How Does Matrix ‘stretch space’? x 1 x 2 b 3 b 2 b 1 Ax=b 1 Input (N dim.) Output (M dim.) u 1 u 2 x 1 ellipsoid (disk) b
  • 5.
    • Sphere of all unit-length x  ellipsoid of b
    • Output ellipsoid’s axes form orthonormal basis vectors U i :
    • Basis vectors U i are columns of U matrix
    • SVD(A) = U SV T columns of U = OUTPUT basis vectors
    SVD finds: ‘Output’ Vectors U , and… x 1 x 2 b 3 b 2 b 1 Ax=b 1 Input (N dim.) Output (M dim.) u 1 u 2 x 1 ellipsoid (disk) b
  • 6. SVD finds: …‘Input’ Vectors V , and…
    • For each U i , make a matching V i that:
      • Transforms to U i (with scaling s i ) : s i (A V i ) = U i
      • Forms an orthonormal basis of input space
    • SVD(A) = US V T columns of V = INPUT basis vectors
    x 1 x 2 Ax=b x Input (N dim.) b 3 b 2 b 1 Output (M dim.) u 1 u 2 v 2 v 1 b
  • 7. SVD finds: …‘Scale’ factors S .
    • We have Unit-length Output U i , vectors,
    • Each from a Unit-length Input V i vector, so
    • So we need ‘scale factor’ (singular values) s i to link input to output: s i (A V i ) = U i
    • SVD(A) = U S V T
    x 1 x 2 Ax=b x Input (N dim.) b 3 b 2 b 1 Output (M dim.) u 1 u 2 v 2 v 1 s 2 s 1 b
  • 8. SVD Review: What is it?
    • Finish: SVD(A) = USV T
      • add ‘ missing ’ U i or V i , define these s i =0 .
      • Singular matrix S : diagonals are s i : ‘v-to-u scale factors’
      • Matrix U , Matrix V have columns U i and V i
    x 1 x 2 Ax=b x Input (N dim.) b 3 b 2 b 1 Output (M dim.) u 1 u 3 u 2 v 2 v 1 A = USV T ‘Find Input & Output Axes, Linked by Scale’ b
  • 9. ‘ Let SVDs Explain it All for You’
      • cool! U and V are Orthonormal! U -1 = U T , V -1 = V T
      • Rank(A)? == # of non-zero singular values s i
      • ‘ ill conditioned’? Some s i are nearly zero
      • ‘ Invert’ a non-square matrix A ? Amazing! pseudo-inverse: A=USV T ; A VS -1 U T = I; so A -1 = V S -1 U T
    x 1 x 2 Ax=b x Input (N dim.) b 3 b 2 b 1 Output (M dim.) u 1 u 3 u 2 v 2 v 1 A = USV T ‘Find Input & Output Axes, Linked by Scale’
  • 10. ‘ Solve for the Null Space’ A x =0
      • Easy! x is the V i axis that doesn’t affect the output ( s i = 0 )
      • are all s i nonzero? then the only answer is x=0.
      • Null ‘space’ because V i is a DIMENSION--- x = Vi * a
      • More than 1 zero-valued s i ? null space >1 dimension ... x = a V i1 + b V i2 + …
    x 1 x 2 Ax=b x Input (N dim.) b 3 b 2 b 1 Output (M dim.) u 1 u 3 u 2 v 2 v 1 A = USV T ‘Find Input & Output Axes, Linked by Scale’
  • 11. More on SVDs: Handout
    • Zisserman Appendix 3, pg 556 (cryptic)
    • Linear Algebra books
    • Good online introduction: (your handout) Leach, S. “Singular Value Decomposition – A primer,” Unpublished Manuscript, Department of Computer Science, Brown University. http://www.cs.brown.edu/research/ai/dynamics/tutorial/Postscript/ http://nextlib.lifegoo.com/user/jack/article/1791
    • Google on Singular Value Decomposition...
  • 12. END