Cs221 linear algebra
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Cs221 linear algebra

on

  • 1,018 views

 

Statistics

Views

Total Views
1,018
Views on SlideShare
1,018
Embed Views
0

Actions

Likes
4
Downloads
87
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Cs221 linear algebra Presentation Transcript

  • 1. Linear Algebra Review CS221: Introduction to Articial Intelligence Carlos Fernandez-Granda 10/11/2011CS221 Linear Algebra Review 10/11/2011 1 / 37
  • 2. Index1 Vectors2 Matrices3 Matrix Decompositions4 Application : Image Compression CS221 Linear Algebra Review 10/11/2011 2 / 37
  • 3. Vectors1 Vectors2 Matrices3 Matrix Decompositions4 Application : Image Compression CS221 Linear Algebra Review 10/11/2011 3 / 37
  • 4. VectorsWhat is a vector ? CS221 Linear Algebra Review 10/11/2011 4 / 37
  • 5. VectorsWhat is a vector ? An ordered tuple of numbers : u   1 u 2  u=  . . .  un CS221 Linear Algebra Review 10/11/2011 4 / 37
  • 6. VectorsWhat is a vector ? An ordered tuple of numbers : u   1 u 2  u=  . . .  un A quantity that has a magnitude and a direction. CS221 Linear Algebra Review 10/11/2011 4 / 37
  • 7. VectorsVector spaces More formally a vector is an element of a vector space. CS221 Linear Algebra Review 10/11/2011 5 / 37
  • 8. VectorsVector spaces More formally a vector is an element of a vector space. A vector space V is a set that contains all linear combinations of its elements : CS221 Linear Algebra Review 10/11/2011 5 / 37
  • 9. VectorsVector spaces More formally a vector is an element of a vector space. A vector space V is a set that contains all linear combinations of its elements : If vectors u and v ∈ V , then u + v ∈ V . CS221 Linear Algebra Review 10/11/2011 5 / 37
  • 10. VectorsVector spaces More formally a vector is an element of a vector space. A vector space V is a set that contains all linear combinations of its elements : If vectors u and v ∈ V , then u + v ∈ V . If vector u ∈ V , then α u ∈ V for any scalar α. CS221 Linear Algebra Review 10/11/2011 5 / 37
  • 11. VectorsVector spaces More formally a vector is an element of a vector space. A vector space V is a set that contains all linear combinations of its elements : If vectors u and v ∈ V , then u + v ∈ V . If vector u ∈ V , then α u ∈ V for any scalar α. There exists 0 ∈ V such that u + 0 = u for any u ∈ V . CS221 Linear Algebra Review 10/11/2011 5 / 37
  • 12. VectorsVector spaces More formally a vector is an element of a vector space. A vector space V is a set that contains all linear combinations of its elements : If vectors u and v ∈ V , then u + v ∈ V . If vector u ∈ V , then α u ∈ V for any scalar α. There exists 0 ∈ V such that u + 0 = u for any u ∈ V . A subspace is a subset of a vector space that is also a vector space. CS221 Linear Algebra Review 10/11/2011 5 / 37
  • 13. VectorsExamples Euclidean space Rn . CS221 Linear Algebra Review 10/11/2011 6 / 37
  • 14. VectorsExamples Euclidean space Rn . The span of any set of vectors {u1 , u2 , . . . , un }, dened as : span (u1 , u2 , . . . , un ) = {α1 u1 + α2 u2 + · · · + αn un | αi ∈ R} CS221 Linear Algebra Review 10/11/2011 6 / 37
  • 15. VectorsSubspaces A line through the origin in Rn (span of a vector). CS221 Linear Algebra Review 10/11/2011 7 / 37
  • 16. VectorsSubspaces A line through the origin in Rn (span of a vector). A plane in Rn (span of two vectors). CS221 Linear Algebra Review 10/11/2011 7 / 37
  • 17. VectorsLinear Independence and Basis of Vector Spaces A vector u is linearly independent of a set of vectors {u1 , u2 , . . . , un } if it does not lie in their span. CS221 Linear Algebra Review 10/11/2011 8 / 37
  • 18. VectorsLinear Independence and Basis of Vector Spaces A vector u is linearly independent of a set of vectors {u1 , u2 , . . . , un } if it does not lie in their span. A set of vectors is linearly independent if every vector is linearly independent of the rest. CS221 Linear Algebra Review 10/11/2011 8 / 37
  • 19. VectorsLinear Independence and Basis of Vector Spaces A vector u is linearly independent of a set of vectors {u1 , u2 , . . . , un } if it does not lie in their span. A set of vectors is linearly independent if every vector is linearly independent of the rest. A basis of a vector space V is a linearly independent set of vectors whose span is equal to V . CS221 Linear Algebra Review 10/11/2011 8 / 37
  • 20. VectorsLinear Independence and Basis of Vector Spaces A vector u is linearly independent of a set of vectors {u1 , u2 , . . . , un } if it does not lie in their span. A set of vectors is linearly independent if every vector is linearly independent of the rest. A basis of a vector space V is a linearly independent set of vectors whose span is equal to V . If a vector space has a basis with d vectors, its dimension is d. CS221 Linear Algebra Review 10/11/2011 8 / 37
  • 21. VectorsLinear Independence and Basis of Vector Spaces A vector u is linearly independent of a set of vectors {u1 , u2 , . . . , un } if it does not lie in their span. A set of vectors is linearly independent if every vector is linearly independent of the rest. A basis of a vector space V is a linearly independent set of vectors whose span is equal to V . If a vector space has a basis with d vectors, its dimension is d. Any other basis of the same space will consist of exactly d vectors. CS221 Linear Algebra Review 10/11/2011 8 / 37
  • 22. VectorsLinear Independence and Basis of Vector Spaces A vector u is linearly independent of a set of vectors {u1 , u2 , . . . , un } if it does not lie in their span. A set of vectors is linearly independent if every vector is linearly independent of the rest. A basis of a vector space V is a linearly independent set of vectors whose span is equal to V . If a vector space has a basis with d vectors, its dimension is d. Any other basis of the same space will consist of exactly d vectors. The dimension of a vector space can be innite (function spaces). CS221 Linear Algebra Review 10/11/2011 8 / 37
  • 23. VectorsNorm of a Vector A norm ||u|| measures the magnitude of a vector. CS221 Linear Algebra Review 10/11/2011 9 / 37
  • 24. VectorsNorm of a Vector A norm ||u|| measures the magnitude of a vector. Mathematically, any function from the vector space to R that satises : CS221 Linear Algebra Review 10/11/2011 9 / 37
  • 25. VectorsNorm of a Vector A norm ||u|| measures the magnitude of a vector. Mathematically, any function from the vector space to R that satises : Homogeneity ||α u|| = |α| ||u|| CS221 Linear Algebra Review 10/11/2011 9 / 37
  • 26. VectorsNorm of a Vector A norm ||u|| measures the magnitude of a vector. Mathematically, any function from the vector space to R that satises : Homogeneity ||α u|| = |α| ||u|| Triangle inequality ||u + v|| ≤ ||u|| + ||v|| CS221 Linear Algebra Review 10/11/2011 9 / 37
  • 27. VectorsNorm of a Vector A norm ||u|| measures the magnitude of a vector. Mathematically, any function from the vector space to R that satises : Homogeneity ||α u|| = |α| ||u|| Triangle inequality ||u + v|| ≤ ||u|| + ||v|| Point separation ||u|| = 0 if and only if u = 0. CS221 Linear Algebra Review 10/11/2011 9 / 37
  • 28. VectorsNorm of a Vector A norm ||u|| measures the magnitude of a vector. Mathematically, any function from the vector space to R that satises : Homogeneity ||α u|| = |α| ||u|| Triangle inequality ||u + v|| ≤ ||u|| + ||v|| Point separation ||u|| = 0 if and only if u = 0. Examples : Manhattan or 1 norm : ||u||1 = i |ui |. Euclidean or 2 norm : ||u||2 = i ui . 2 CS221 Linear Algebra Review 10/11/2011 9 / 37
  • 29. VectorsInner Product Inner product between u and v : u, v = n uv. i =1 i i CS221 Linear Algebra Review 10/11/2011 10 / 37
  • 30. VectorsInner Product Inner product between u and v : u, v = n=1 ui vi . i It is the projection of one vector onto the other. CS221 Linear Algebra Review 10/11/2011 10 / 37
  • 31. VectorsInner Product Inner product between u and v : u, v = n=1 ui vi . i It is the projection of one vector onto the other. Related to the Euclidean norm : u, u = ||u||2 . 2 CS221 Linear Algebra Review 10/11/2011 10 / 37
  • 32. VectorsInner ProductThe inner product is a measure of correlation between two vectors, scaledby the norms of the vectors : CS221 Linear Algebra Review 10/11/2011 11 / 37
  • 33. VectorsInner ProductThe inner product is a measure of correlation between two vectors, scaledby the norms of the vectors : If u, v > 0, u and v are aligned. CS221 Linear Algebra Review 10/11/2011 11 / 37
  • 34. VectorsInner ProductThe inner product is a measure of correlation between two vectors, scaledby the norms of the vectors : If u, v > 0, u and v are aligned. If u, v < 0, u and v are opposed. CS221 Linear Algebra Review 10/11/2011 11 / 37
  • 35. VectorsInner ProductThe inner product is a measure of correlation between two vectors, scaledby the norms of the vectors : If u, v > 0, u and v are aligned. If u, v < 0, u and v are opposed. If u, v = 0, u and v are orthogonal. CS221 Linear Algebra Review 10/11/2011 11 / 37
  • 36. VectorsOrthonormal Basis The vectors in an orthonormal basis have unit Euclidean norm and are orthogonal to each other. CS221 Linear Algebra Review 10/11/2011 12 / 37
  • 37. VectorsOrthonormal Basis The vectors in an orthonormal basis have unit Euclidean norm and are orthogonal to each other. To express a vector x in an orthonormal basis, we can just take inner products. CS221 Linear Algebra Review 10/11/2011 12 / 37
  • 38. VectorsOrthonormal Basis The vectors in an orthonormal basis have unit Euclidean norm and are orthogonal to each other. To express a vector x in an orthonormal basis, we can just take inner products. Example : x = α1 b1 + α2 b2 . CS221 Linear Algebra Review 10/11/2011 12 / 37
  • 39. VectorsOrthonormal Basis The vectors in an orthonormal basis have unit Euclidean norm and are orthogonal to each other. To express a vector x in an orthonormal basis, we can just take inner products. Example : x = α1 b1 + α2 b2 . We compute x, b1 : x, b1 = α1 b1 + α2 b2 , b1 = α1 b1 , b1 + α2 b2 , b1 By linearity. = α1 + 0 By orthonormality. CS221 Linear Algebra Review 10/11/2011 12 / 37
  • 40. VectorsOrthonormal Basis The vectors in an orthonormal basis have unit Euclidean norm and are orthogonal to each other. To express a vector x in an orthonormal basis, we can just take inner products. Example : x = α1 b1 + α2 b2 . We compute x, b1 : x, b1 = α1 b1 + α2 b2 , b1 = α1 b1 , b1 + α2 b2 , b1 By linearity. = α1 + 0 By orthonormality. Likewise, α2 = x, b2 . CS221 Linear Algebra Review 10/11/2011 12 / 37
  • 41. Matrices1 Vectors2 Matrices3 Matrix Decompositions4 Application : Image Compression CS221 Linear Algebra Review 10/11/2011 13 / 37
  • 42. MatricesLinear Operator A linear operator L : U → V is a map from a vector space U to another vector space V that satises : CS221 Linear Algebra Review 10/11/2011 14 / 37
  • 43. MatricesLinear Operator A linear operator L : U → V is a map from a vector space U to another vector space V that satises : L (u1 + u2 ) = L (u1 ) + L (u2 ) . CS221 Linear Algebra Review 10/11/2011 14 / 37
  • 44. MatricesLinear Operator A linear operator L : U → V is a map from a vector space U to another vector space V that satises : . L (u1 + u2 ) = L (u1 ) + L (u2 ) L (α u) = α L (u) for any scalar α. CS221 Linear Algebra Review 10/11/2011 14 / 37
  • 45. MatricesLinear Operator A linear operator L : U → V is a map from a vector space U to another vector space V that satises : . L (u1 + u2 ) = L (u1 ) + L (u2 ) L (α u) = α L (u) for any scalar α. If the dimension n of U and m of V are nite, L can be represented by an m × n matrix A. A A An   11 12 ... 1 A 21 A 22 ... A n 2 A=   ...  Am 1 Am 2 ... Amn CS221 Linear Algebra Review 10/11/2011 14 / 37
  • 46. MatricesMatrix Vector Multiplication A A An u v      11 12 ... 1 1 1 A 21 A 22 ... A nu 2 2  v 2   Au =   =  ...  . . .  . . .  Am 1 Am 2 ... Amn un vm CS221 Linear Algebra Review 10/11/2011 15 / 37
  • 47. MatricesMatrix Vector Multiplication A A An u v      11 12 ... 1 1 1 A 21 A 22 ... A nu 2 2  v 2   Au =   =  ...  . . .  . . .  Am 1 Am 2 ... Amn un vm CS221 Linear Algebra Review 10/11/2011 15 / 37
  • 48. MatricesMatrix Vector Multiplication A A An u v      11 12 ... 1 1 1 A 21 A 22 ... A nu 2 2  v 2   Au =   =  ...  . . .  . . .  Am 1 Am 2 ... Amn un vm CS221 Linear Algebra Review 10/11/2011 15 / 37
  • 49. MatricesMatrix Vector Multiplication A A An u v      11 12 ... 1 1 1 A 21 A 22 ... A nu 2 2  v 2   Au =   =  ...  . . .  . . .  Am 1 Am 2 ... Amn un vm CS221 Linear Algebra Review 10/11/2011 15 / 37
  • 50. MatricesMatrix ProductApplying two linear operators A and B denes another linear operator,which can be represented by another matrix AB. A A An B B Bp    11 12 ... 1 11 12 ... 1 A 21 A 22 ...  B A n 2 21 B22 ... B p 2 AB =    ...  ...  Am Am 1 2 ... Amn Bn 1 Bn 2 ... Bnp AB AB . . . AB p   11 12 1  AB AB 21 22 . . . AB p  2 =   ...  ABm 1 ABm 2 ... ABmpNote that if A is m × n and B is n × p, then AB is m × p. CS221 Linear Algebra Review 10/11/2011 16 / 37
  • 51. MatricesMatrix ProductApplying two linear operators A and B denes another linear operator,which can be represented by another matrix AB. A A An B B Bp    11 12 ... 1 11 12 ... 1 A 21 A 22 ...  B A n 2 21 B22 ... B p 2 AB =    ...  ...  Am Am 1 2 ... Amn Bn 1 Bn 2 ... Bnp AB AB . . . AB p   11 12 1  AB AB 21 22 . . . AB p  2 =   ...  ABm 1 ABm 2 ... ABmpNote that if A is m × n and B is n × p, then AB is m × p. CS221 Linear Algebra Review 10/11/2011 16 / 37
  • 52. MatricesMatrix ProductApplying two linear operators A and B denes another linear operator,which can be represented by another matrix AB. A A An B B Bp    11 12 ... 1 11 12 ... 1 A 21 A 22 ...  B A n 2 21 B22 ... B p 2 AB =    ...  ...  Am Am 1 2 ... Amn Bn 1 Bn 2 ... Bnp AB AB . . . AB p   11 12 1  AB AB 21 22 . . . AB p  2 =   ...  ABm 1 ABm 2 ... ABmpNote that if A is m × n and B is n × p, then AB is m × p. CS221 Linear Algebra Review 10/11/2011 16 / 37
  • 53. MatricesMatrix ProductApplying two linear operators A and B denes another linear operator,which can be represented by another matrix AB. A A An B B Bp    11 12 ... 1 11 12 ... 1 A 21 A 22 ...  B A n 2 21 B22 ... B p 2 AB =    ...  ...  Am Am 1 2 ... Amn Bn 1 Bn 2 ... Bnp AB AB . . . AB p   11 12 1  AB AB 21 22 . . . AB p  2 =   ...  ABm 1 ABm 2 ... ABmpNote that if A is m × n and B is n × p, then AB is m × p. CS221 Linear Algebra Review 10/11/2011 16 / 37
  • 54. MatricesMatrix ProductApplying two linear operators A and B denes another linear operator,which can be represented by another matrix AB. A A An B B Bp    11 12 ... 1 11 12 ... 1 A 21 A 22 ...  B A n 2 21 B22 ... B p 2 AB =    ...  ...  Am Am 1 2 ... Amn Bn 1 Bn 2 ... Bnp AB AB . . . AB p   11 12 1  AB AB 21 22 . . . AB p  2 =   ...  ABm 1 ABm 2 ... ABmpNote that if A is m × n and B is n × p, then AB is m × p. CS221 Linear Algebra Review 10/11/2011 16 / 37
  • 55. MatricesMatrix ProductApplying two linear operators A and B denes another linear operator,which can be represented by another matrix AB. A A An B B Bp    11 12 ... 1 11 12 ... 1 A 21 A 22 ...  B A n 2 21 B22 ... B p 2 AB =    ...  ...  Am Am 1 2 ... Amn Bn 1 Bn 2 ... Bnp AB AB . . . AB p   11 12 1  AB AB 21 22 . . . AB p  2 =   ...  ABm 1 ABm 2 ... ABmpNote that if A is m × n and B is n × p, then AB is m × p. CS221 Linear Algebra Review 10/11/2011 16 / 37
  • 56. MatricesMatrix ProductApplying two linear operators A and B denes another linear operator,which can be represented by another matrix AB. A A An B B Bp    11 12 ... 1 11 12 ... 1 A 21 A 22 ...  B A n 2 21 B22 ... B p 2 AB =    ...  ...  Am Am 1 2 ... Amn Bn 1 Bn 2 ... Bnp AB AB . . . AB p   11 12 1  AB AB 21 22 . . . AB p  2 =   ...  ABm 1 ABm 2 ... ABmpNote that if A is m × n and B is n × p, then AB is m × p. CS221 Linear Algebra Review 10/11/2011 16 / 37
  • 57. MatricesMatrix ProductApplying two linear operators A and B denes another linear operator,which can be represented by another matrix AB. A A An B B Bp    11 12 ... 1 11 12 ... 1 A 21 A 22 ...  B A n 2 21 B22 ... B p 2 AB =    ...  ...  Am Am 1 2 ... Amn Bn 1 Bn 2 ... Bnp AB AB . . . AB p   11 12 1  AB AB 21 22 . . . AB p  2 =   ...  ABm 1 ABm 2 ... ABmpNote that if A is m × n and B is n × p, then AB is m × p. CS221 Linear Algebra Review 10/11/2011 16 / 37
  • 58. MatricesMatrix ProductApplying two linear operators A and B denes another linear operator,which can be represented by another matrix AB. A A An B B Bp    11 12 ... 1 11 12 ... 1 A 21 A 22 ...  B A n 2 21 B22 ... B p 2 AB =    ...  ...  Am Am 1 2 ... Amn Bn 1 Bn 2 ... Bnp AB AB . . . AB p   11 12 1  AB AB 21 22 . . . AB p  2 =   ...  ABm 1 ABm 2 ... ABmpNote that if A is m × n and B is n × p, then AB is m × p. CS221 Linear Algebra Review 10/11/2011 16 / 37
  • 59. MatricesMatrix ProductApplying two linear operators A and B denes another linear operator,which can be represented by another matrix AB. A A An B B Bp    11 12 ... 1 11 12 ... 1 A 21 A 22 ...  B A n 2 21 B22 ... B p 2 AB =    ...  ...  Am Am 1 2 ... Amn Bn 1 Bn 2 ... Bnp AB AB . . . AB p   11 12 1  AB AB 21 22 . . . AB p  2 =   ...  ABm 1 ABm 2 ... ABmpNote that if A is m × n and B is n × p, then AB is m × p. CS221 Linear Algebra Review 10/11/2011 16 / 37
  • 60. MatricesTranspose of a Matrix The transpose of a matrix is obtained by ipping the rows and columns. A11 A21 . . . An1    A12 A22 . . . An2  AT =    ...  A1m A2m . . . Anm CS221 Linear Algebra Review 10/11/2011 17 / 37
  • 61. MatricesTranspose of a Matrix The transpose of a matrix is obtained by ipping the rows and columns. A11 A21 . . . An1    A12 A22 . . . An2  AT =    ...  A1m A2m . . . Anm Some simple properties : CS221 Linear Algebra Review 10/11/2011 17 / 37
  • 62. MatricesTranspose of a Matrix The transpose of a matrix is obtained by ipping the rows and columns. A11 A21 . . . An1    A12 A22 . . . An2  AT =    ...  A1m A2m . . . Anm Some simple properties : T AT =A CS221 Linear Algebra Review 10/11/2011 17 / 37
  • 63. MatricesTranspose of a Matrix The transpose of a matrix is obtained by ipping the rows and columns. A11 A21 . . . An1    A12 A22 . . . An2  AT =    ...  A1m A2m . . . Anm Some simple properties : T AT = A T (A + B) = AT + BT CS221 Linear Algebra Review 10/11/2011 17 / 37
  • 64. MatricesTranspose of a Matrix The transpose of a matrix is obtained by ipping the rows and columns. A11 A21 . . . An1    A12 A22 . . . An2  AT =    ...  A1m A2m . . . Anm Some simple properties : T AT = A T (A + B) = AT + BT T (AB) = BT AT CS221 Linear Algebra Review 10/11/2011 17 / 37
  • 65. MatricesTranspose of a Matrix The transpose of a matrix is obtained by ipping the rows and columns. A11 A21 . . . An1    A12 A22 . . . An2  AT =    ...  A1m A2m . . . Anm Some simple properties : T AT = A T (A + B) = AT + BT T (AB) = BT AT The inner product for vectors can be represented as u, v = uT v. CS221 Linear Algebra Review 10/11/2011 17 / 37
  • 66. MatricesIdentity Operator 1 0 ... 0   0 1 . . . 0 I=   ...  0 0 ... 1 CS221 Linear Algebra Review 10/11/2011 18 / 37
  • 67. MatricesIdentity Operator 1 0 ... 0   0 1 . . . 0 I=   ...  0 0 ... 1 For any matrix A, AI = A. CS221 Linear Algebra Review 10/11/2011 18 / 37
  • 68. MatricesIdentity Operator 1 0 ... 0   0 1 . . . 0 I=   ...  0 0 ... 1 For any matrix A, AI = A. I is the identity operator for the matrix product. CS221 Linear Algebra Review 10/11/2011 18 / 37
  • 69. MatricesColumn Space, Row Space and Rank Let A be an m × n matrix. CS221 Linear Algebra Review 10/11/2011 19 / 37
  • 70. MatricesColumn Space, Row Space and Rank Let A be an m × n matrix. Column space : CS221 Linear Algebra Review 10/11/2011 19 / 37
  • 71. MatricesColumn Space, Row Space and Rank Let A be an m × n matrix. Column space : Span of the columns of A. CS221 Linear Algebra Review 10/11/2011 19 / 37
  • 72. MatricesColumn Space, Row Space and Rank Let A be an m × n matrix. Column space : Span of the columns of A. Linear subspace of Rm . CS221 Linear Algebra Review 10/11/2011 19 / 37
  • 73. MatricesColumn Space, Row Space and Rank Let A be an m × n matrix. Column space : Span of the columns of A. Linear subspace of Rm . Row space : CS221 Linear Algebra Review 10/11/2011 19 / 37
  • 74. MatricesColumn Space, Row Space and Rank Let A be an m × n matrix. Column space : Span of the columns of A. Linear subspace of Rm . Row space : Span of the rows of A. CS221 Linear Algebra Review 10/11/2011 19 / 37
  • 75. MatricesColumn Space, Row Space and Rank Let A be an m × n matrix. Column space : Span of the columns of A. Linear subspace of Rm . Row space : Span of the rows of A. Linear subspace of Rn . CS221 Linear Algebra Review 10/11/2011 19 / 37
  • 76. MatricesColumn Space, Row Space and Rank Let A be an m × n matrix. Column space : Span of the columns of A. Linear subspace of Rm . Row space : Span of the rows of A. Linear subspace of Rn . Important fact : The column and row spaces of any matrix have the same dimension. CS221 Linear Algebra Review 10/11/2011 19 / 37
  • 77. MatricesColumn Space, Row Space and Rank Let A be an m × n matrix. Column space : Span of the columns of A. Linear subspace of Rm . Row space : Span of the rows of A. Linear subspace of Rn . Important fact : The column and row spaces of any matrix have the same dimension. This dimension is the rank of the matrix. CS221 Linear Algebra Review 10/11/2011 19 / 37
  • 78. MatricesRange and Null space Let A be an m × n matrix. CS221 Linear Algebra Review 10/11/2011 20 / 37
  • 79. MatricesRange and Null space Let A be an m × n matrix. Range : CS221 Linear Algebra Review 10/11/2011 20 / 37
  • 80. MatricesRange and Null space Let A be an m × n matrix. Range : Set of vectors equal to Au for some u ∈ Rn . CS221 Linear Algebra Review 10/11/2011 20 / 37
  • 81. MatricesRange and Null space Let A be an m × n matrix. Range : Set of vectors equal to Au for some u ∈ Rn . Linear subspace of Rm . CS221 Linear Algebra Review 10/11/2011 20 / 37
  • 82. MatricesRange and Null space Let A be an m × n matrix. Range : Set of vectors equal to Au for some u ∈ Rn . Linear subspace of Rm . Null space : CS221 Linear Algebra Review 10/11/2011 20 / 37
  • 83. MatricesRange and Null space Let A be an m × n matrix. Range : Set of vectors equal to Au for some u ∈ Rn . Linear subspace of Rm . Null space : u belongs to the null space if Au = 0. CS221 Linear Algebra Review 10/11/2011 20 / 37
  • 84. MatricesRange and Null space Let A be an m × n matrix. Range : Set of vectors equal to Au for some u ∈ Rn . Linear subspace of Rm . Null space : u belongs to the null space if Au = 0. Subspace of Rn . CS221 Linear Algebra Review 10/11/2011 20 / 37
  • 85. MatricesRange and Null space Let A be an m × n matrix. Range : Set of vectors equal to Au for some u ∈ Rn . Linear subspace of Rm . Null space : u belongs to the null space if Au = 0. Subspace of Rn . Every vector in the null space is orthogonal to the rows of A. The null space and row space of a matrix are orthogonal. CS221 Linear Algebra Review 10/11/2011 20 / 37
  • 86. MatricesRange and Column SpaceAnother interpretation of the matrix vector product (Ai is column i of A) : u   1 u 2   Au = A1 A2 ... An  . . .  un = u1 A1 + u2 A2 + · · · + un An CS221 Linear Algebra Review 10/11/2011 21 / 37
  • 87. MatricesRange and Column SpaceAnother interpretation of the matrix vector product (Ai is column i of A) : u   1 u 2   Au = A1 A2 ... An  . . .  un = u1 A1 + u2 A2 + · · · + un An The result is a linear combination of the columns of A. CS221 Linear Algebra Review 10/11/2011 21 / 37
  • 88. MatricesRange and Column SpaceAnother interpretation of the matrix vector product (Ai is column i of A) : u   1 u 2   Au = A1 A2 ... An  . . .  un = u1 A1 + u2 A2 + · · · + un An The result is a linear combination of the columns of A. For any matrix, the range is equal to the column space. CS221 Linear Algebra Review 10/11/2011 21 / 37
  • 89. MatricesMatrix Inverse For an n × n matrix A : rank + dim(null space) = n. CS221 Linear Algebra Review 10/11/2011 22 / 37
  • 90. MatricesMatrix Inverse For an n × n matrix A : rank + dim(null space) = n. If dim(null space)=0 then A is full rank. CS221 Linear Algebra Review 10/11/2011 22 / 37
  • 91. MatricesMatrix Inverse For an n × n matrix A : rank + dim(null space) = n. If dim(null space)=0 then A is full rank. In this case, the action of the matrix is invertible. CS221 Linear Algebra Review 10/11/2011 22 / 37
  • 92. MatricesMatrix Inverse For an n × n matrix A : rank + dim(null space) = n. If dim(null space)=0 then A is full rank. In this case, the action of the matrix is invertible. The inversion is also linear and consequently can be represented by another matrix A−1 . CS221 Linear Algebra Review 10/11/2011 22 / 37
  • 93. MatricesMatrix Inverse For an n × n matrix A : rank + dim(null space) = n. If dim(null space)=0 then A is full rank. In this case, the action of the matrix is invertible. The inversion is also linear and consequently can be represented by another matrix A−1 . A−1 is the only matrix such that A−1 A = AA−1 = I. CS221 Linear Algebra Review 10/11/2011 22 / 37
  • 94. MatricesOrthogonal Matrices An orthogonal matrix U satises UT U = I. CS221 Linear Algebra Review 10/11/2011 23 / 37
  • 95. MatricesOrthogonal Matrices An orthogonal matrix U satises UT U = I. Equivalently, U has orthonormal columns. CS221 Linear Algebra Review 10/11/2011 23 / 37
  • 96. MatricesOrthogonal Matrices An orthogonal matrix U satises UT U = I. Equivalently, U has orthonormal columns. Applying an orthogonal matrix to two vectors does not change their inner product : Uu, Uv = (Uu) Uv T = uT UT Uv = uT v = u, v CS221 Linear Algebra Review 10/11/2011 23 / 37
  • 97. MatricesOrthogonal Matrices An orthogonal matrix U satises UT U = I. Equivalently, U has orthonormal columns. Applying an orthogonal matrix to two vectors does not change their inner product : Uu, Uv = (Uu) Uv T = uT UT Uv = uT v = u, v Example : Matrices that represent rotations are orthogonal. CS221 Linear Algebra Review 10/11/2011 23 / 37
  • 98. MatricesTrace and Determinant The trace is the sum of the diagonal elements of a square matrix. CS221 Linear Algebra Review 10/11/2011 24 / 37
  • 99. MatricesTrace and Determinant The trace is the sum of the diagonal elements of a square matrix. The determinant of a square matrix A is denoted by |A|. CS221 Linear Algebra Review 10/11/2011 24 / 37
  • 100. MatricesTrace and Determinant The trace is the sum of the diagonal elements of a square matrix. The determinant of a square matrix A is denoted by |A|. ab For a 2 × 2 matrix A = c d : CS221 Linear Algebra Review 10/11/2011 24 / 37
  • 101. MatricesTrace and Determinant The trace is the sum of the diagonal elements of a square matrix. The determinant of a square matrix A is denoted by |A|. ab For a 2 × 2 matrix A = c d : |A| = ad − bc CS221 Linear Algebra Review 10/11/2011 24 / 37
  • 102. MatricesTrace and Determinant The trace is the sum of the diagonal elements of a square matrix. The determinant of a square matrix A is denoted by |A|. ab For a 2 × 2 matrix A = c d : |A| = ad − bc The absolute value of |A| is the area of the parallelogram given by the rows of A. CS221 Linear Algebra Review 10/11/2011 24 / 37
  • 103. MatricesProperties of the Determinant The denition can be generalized to larger matrices. CS221 Linear Algebra Review 10/11/2011 25 / 37
  • 104. MatricesProperties of the Determinant The denition can be generalized to larger matrices. |A| = AT . CS221 Linear Algebra Review 10/11/2011 25 / 37
  • 105. MatricesProperties of the Determinant The denition can be generalized to larger matrices. |A| = AT . |AB| = |A| |B| CS221 Linear Algebra Review 10/11/2011 25 / 37
  • 106. MatricesProperties of the Determinant The denition can be generalized to larger matrices. |A| = AT . |AB| = |A| |B| |A| = 0 if and only if A is not invertible. CS221 Linear Algebra Review 10/11/2011 25 / 37
  • 107. MatricesProperties of the Determinant The denition can be generalized to larger matrices. |A| = AT . |AB| = |A| |B| |A| = 0 if and only if A is not invertible. If A is invertible, then A−1 = |A | . 1 CS221 Linear Algebra Review 10/11/2011 25 / 37
  • 108. Matrix Decompositions1 Vectors2 Matrices3 Matrix Decompositions4 Application : Image Compression CS221 Linear Algebra Review 10/11/2011 26 / 37
  • 109. Matrix DecompositionsEigenvalues and Eigenvectors An eigenvalue λ of a square matrix A satises : Au = λu for some vector u, which we call an eigenvector. CS221 Linear Algebra Review 10/11/2011 27 / 37
  • 110. Matrix DecompositionsEigenvalues and Eigenvectors An eigenvalue λ of a square matrix A satises : Au = λu for some vector u, which we call an eigenvector. Geometrically, the operator A expands (λ > 1) or contracts (λ < 1) eigenvectors but does not rotate them. CS221 Linear Algebra Review 10/11/2011 27 / 37
  • 111. Matrix DecompositionsEigenvalues and Eigenvectors An eigenvalue λ of a square matrix A satises : Au = λu for some vector u, which we call an eigenvector. Geometrically, the operator A expands (λ > 1) or contracts (λ < 1) eigenvectors but does not rotate them. If u is an eigenvector of A, it is in the null space of A − λI, which is consequently not invertible. CS221 Linear Algebra Review 10/11/2011 27 / 37
  • 112. Matrix DecompositionsEigenvalues and Eigenvectors An eigenvalue λ of a square matrix A satises : Au = λu for some vector u, which we call an eigenvector. Geometrically, the operator A expands (λ > 1) or contracts (λ < 1) eigenvectors but does not rotate them. If u is an eigenvector of A, it is in the null space of A − λI, which is consequently not invertible. The eigenvalues of A are the roots of the equation |A − λI| = 0. CS221 Linear Algebra Review 10/11/2011 27 / 37
  • 113. Matrix DecompositionsEigenvalues and Eigenvectors An eigenvalue λ of a square matrix A satises : Au = λu for some vector u, which we call an eigenvector. Geometrically, the operator A expands (λ > 1) or contracts (λ < 1) eigenvectors but does not rotate them. If u is an eigenvector of A, it is in the null space of A − λI, which is consequently not invertible. The eigenvalues of A are the roots of the equation |A − λI| = 0. Not the way eigenvalues are calculated in practice. CS221 Linear Algebra Review 10/11/2011 27 / 37
  • 114. Matrix DecompositionsEigenvalues and Eigenvectors An eigenvalue λ of a square matrix A satises : Au = λu for some vector u, which we call an eigenvector. Geometrically, the operator A expands (λ > 1) or contracts (λ < 1) eigenvectors but does not rotate them. If u is an eigenvector of A, it is in the null space of A − λI, which is consequently not invertible. The eigenvalues of A are the roots of the equation |A − λI| = 0. Not the way eigenvalues are calculated in practice. Eigenvalues and eigenvectors can be complex valued, even if all the entries of A are real. CS221 Linear Algebra Review 10/11/2011 27 / 37
  • 115. Matrix DecompositionsEigendecomposition of a Matrix Let A be an n × n square matrix with n linearly independent eigenvectors p1 . . . pn and eigenvalues λ1 . . . λn . CS221 Linear Algebra Review 10/11/2011 28 / 37
  • 116. Matrix DecompositionsEigendecomposition of a Matrix Let A be an n × n square matrix with n linearly independent eigenvectors p1 . . . pn and eigenvalues λ1 . . . λn . We dene the matrices : P= p1 . . . pn λ1 0 . . . 0    0 λ2 . . . 0 Λ=   ...  0 0 ... λn CS221 Linear Algebra Review 10/11/2011 28 / 37
  • 117. Matrix DecompositionsEigendecomposition of a Matrix Let A be an n × n square matrix with n linearly independent eigenvectors p1 . . . pn and eigenvalues λ1 . . . λn . We dene the matrices : P= p1 . . . pn λ1 0 . . . 0    0 λ2 . . . 0 Λ=   ...  0 0 ... λn A satises AP = PΛ. CS221 Linear Algebra Review 10/11/2011 28 / 37
  • 118. Matrix DecompositionsEigendecomposition of a Matrix Let A be an n × n square matrix with n linearly independent eigenvectors p1 . . . pn and eigenvalues λ1 . . . λn . We dene the matrices : P= p1 . . . pn λ1 0 . . . 0    0 λ2 . . . 0 Λ=   ...  0 0 ... λn A satises AP = PΛ. P is full rank. Inverting it yields the eigendecomposition of A : A = PΛ P− 1 CS221 Linear Algebra Review 10/11/2011 28 / 37
  • 119. Matrix DecompositionsProperties of the Eigendecomposition Not all matrices are diagonalizable (have an eigendecomposition). Example : ( 1 1 ). 0 1 CS221 Linear Algebra Review 10/11/2011 29 / 37
  • 120. Matrix DecompositionsProperties of the Eigendecomposition Not all matrices are diagonalizable (have an eigendecomposition). Example : ( 1 1 ). 0 1 Trace(A) = Trace(Λ) = n=1 λi . i CS221 Linear Algebra Review 10/11/2011 29 / 37
  • 121. Matrix DecompositionsProperties of the Eigendecomposition Not all matrices are diagonalizable (have an eigendecomposition). Example : ( 1 1 ). 0 1 Trace(A) = Trace(Λ) = n=1 λi . i |A| = |Λ| = n=1 λi . i CS221 Linear Algebra Review 10/11/2011 29 / 37
  • 122. Matrix DecompositionsProperties of the Eigendecomposition Not all matrices are diagonalizable (have an eigendecomposition). Example : ( 1 1 ). 0 1 Trace(A) = Trace(Λ) = n=1 λi . i |A| = |Λ| = n=1 λi . i The rank of A is equal to the number of nonzero eigenvalues. CS221 Linear Algebra Review 10/11/2011 29 / 37
  • 123. Matrix DecompositionsProperties of the Eigendecomposition Not all matrices are diagonalizable (have an eigendecomposition). Example : ( 1 1 ). 0 1 Trace(A) = Trace(Λ) = n=1 λi . i |A| = |Λ| = n=1 λi . i The rank of A is equal to the number of nonzero eigenvalues. If λ is a nonzero eigenvalue of A, λ is an eigenvalue of A−1 with the 1 same eigenvector. CS221 Linear Algebra Review 10/11/2011 29 / 37
  • 124. Matrix DecompositionsProperties of the Eigendecomposition Not all matrices are diagonalizable (have an eigendecomposition). Example : ( 1 1 ). 0 1 Trace(A) = Trace(Λ) = n=1 λi . i |A| = |Λ| = n=1 λi . i The rank of A is equal to the number of nonzero eigenvalues. If λ is a nonzero eigenvalue of A, λ is an eigenvalue of A−1 with the 1 same eigenvector. The eigendecomposition makes it possible to compute matrix powers eciently : m Am = PΛP−1 = PΛP−1 PΛP−1 . . . PΛP−1 = PΛm P−1 . CS221 Linear Algebra Review 10/11/2011 29 / 37
  • 125. Matrix DecompositionsMatlab Example CS221 Linear Algebra Review 10/11/2011 30 / 37
  • 126. Matrix DecompositionsEigendecomposition of a Symmetric Matrix If A = AT then A is symmetric. CS221 Linear Algebra Review 10/11/2011 31 / 37
  • 127. Matrix DecompositionsEigendecomposition of a Symmetric Matrix If A = AT then A is symmetric. The eigenvalues of symmetric matrices are real. CS221 Linear Algebra Review 10/11/2011 31 / 37
  • 128. Matrix DecompositionsEigendecomposition of a Symmetric Matrix If A = AT then A is symmetric. The eigenvalues of symmetric matrices are real. The eigenvectors of symmetric matrices are orthonormal. CS221 Linear Algebra Review 10/11/2011 31 / 37
  • 129. Matrix DecompositionsEigendecomposition of a Symmetric Matrix If A = AT then A is symmetric. The eigenvalues of symmetric matrices are real. The eigenvectors of symmetric matrices are orthonormal. Consequently, the eigendecomposition becomes : A = UΛUT for Λ real and U orthogonal. CS221 Linear Algebra Review 10/11/2011 31 / 37
  • 130. Matrix DecompositionsEigendecomposition of a Symmetric Matrix If A = AT then A is symmetric. The eigenvalues of symmetric matrices are real. The eigenvectors of symmetric matrices are orthonormal. Consequently, the eigendecomposition becomes : A = UΛUT for Λ real and U orthogonal. The eigenvectors of A are an orthonormal basis for the column space (and the row space, since they are equal). CS221 Linear Algebra Review 10/11/2011 31 / 37
  • 131. Matrix DecompositionsEigendecomposition of a Symmetric Matrix The action of a symmetric matrix on a vector Au = UΛUT u can be decomposed into : CS221 Linear Algebra Review 10/11/2011 32 / 37
  • 132. Matrix DecompositionsEigendecomposition of a Symmetric Matrix The action of a symmetric matrix on a vector Au = UΛUT u can be decomposed into : 1 Projection of u onto the column space of A (multiplication by UT ). CS221 Linear Algebra Review 10/11/2011 32 / 37
  • 133. Matrix DecompositionsEigendecomposition of a Symmetric Matrix The action of a symmetric matrix on a vector Au = UΛUT u can be decomposed into : 1 Projection of u onto the column space of A (multiplication by UT ). 2 Scaling of each coecient Ui , u by the corresponding eigenvalue (multiplication by Λ). CS221 Linear Algebra Review 10/11/2011 32 / 37
  • 134. Matrix DecompositionsEigendecomposition of a Symmetric Matrix The action of a symmetric matrix on a vector Au = UΛUT u can be decomposed into : 1 Projection of u onto the column space of A (multiplication by UT ). 2 Scaling of each coecient Ui , u by the corresponding eigenvalue (multiplication by Λ). 3 Linear combination of the eigenvectors scaled by the resulting coecient (multiplication by U). CS221 Linear Algebra Review 10/11/2011 32 / 37
  • 135. Matrix DecompositionsEigendecomposition of a Symmetric Matrix The action of a symmetric matrix on a vector Au = UΛUT u can be decomposed into : 1 Projection of u onto the column space of A (multiplication by UT ). 2 Scaling of each coecient Ui , u by the corresponding eigenvalue (multiplication by Λ). 3 Linear combination of the eigenvectors scaled by the resulting coecient (multiplication by U). Summarizing : n Au = λi Ui , u Ui i =1 CS221 Linear Algebra Review 10/11/2011 32 / 37
  • 136. Matrix DecompositionsEigendecomposition of a Symmetric Matrix The action of a symmetric matrix on a vector Au = UΛUT u can be decomposed into : 1 Projection of u onto the column space of A (multiplication by UT ). 2 Scaling of each coecient Ui , u by the corresponding eigenvalue (multiplication by Λ). 3 Linear combination of the eigenvectors scaled by the resulting coecient (multiplication by U). Summarizing : n Au = λi Ui , u Ui i =1 It would be great to generalize this to all matrices... CS221 Linear Algebra Review 10/11/2011 32 / 37
  • 137. Matrix DecompositionsSingular Value Decomposition Every matrix has a singular value decomposition : A = UΣVT CS221 Linear Algebra Review 10/11/2011 33 / 37
  • 138. Matrix DecompositionsSingular Value Decomposition Every matrix has a singular value decomposition : A = UΣVT The columns of U are an orthonormal basis for the column space of A. CS221 Linear Algebra Review 10/11/2011 33 / 37
  • 139. Matrix DecompositionsSingular Value Decomposition Every matrix has a singular value decomposition : A = UΣVT The columns of U are an orthonormal basis for the column space of A. The columns of V are an orthonormal basis for the row space of A. CS221 Linear Algebra Review 10/11/2011 33 / 37
  • 140. Matrix DecompositionsSingular Value Decomposition Every matrix has a singular value decomposition : A = UΣVT The columns of U are an orthonormal basis for the column space of A. The columns of V are an orthonormal basis for the row space of A. Σ is diagonal. The entries on its diagonal σi = Σii are positive real numbers, called the singular values of A. CS221 Linear Algebra Review 10/11/2011 33 / 37
  • 141. Matrix DecompositionsSingular Value Decomposition Every matrix has a singular value decomposition : A = UΣVT The columns of U are an orthonormal basis for the column space of A. The columns of V are an orthonormal basis for the row space of A. Σ is diagonal. The entries on its diagonal σi = Σii are positive real numbers, called the singular values of A. The action of A on a vector u can be decomposed into : n Au = σi Vi , u Ui i =1 CS221 Linear Algebra Review 10/11/2011 33 / 37
  • 142. Matrix DecompositionsProperties of the Singular Value Decomposition The eigenvalues of the symmetric matrix AT A are equal to the square of the singular values of A : AT A = VΣUT UΣVT = VΣ2 VT CS221 Linear Algebra Review 10/11/2011 34 / 37
  • 143. Matrix DecompositionsProperties of the Singular Value Decomposition The eigenvalues of the symmetric matrix AT A are equal to the square of the singular values of A : AT A = VΣUT UΣVT = VΣ2 VT The rank of a matrix is equal to the number of nonzero singular values. CS221 Linear Algebra Review 10/11/2011 34 / 37
  • 144. Matrix DecompositionsProperties of the Singular Value Decomposition The eigenvalues of the symmetric matrix AT A are equal to the square of the singular values of A : AT A = VΣUT UΣVT = VΣ2 VT The rank of a matrix is equal to the number of nonzero singular values. The largest singular value σ1 is the solution to the optimization problem : ||Ax||2 σ1 = max x=0 ||x||2 CS221 Linear Algebra Review 10/11/2011 34 / 37
  • 145. Matrix DecompositionsProperties of the Singular Value Decomposition The eigenvalues of the symmetric matrix AT A are equal to the square of the singular values of A : AT A = VΣUT UΣVT = VΣ2 VT The rank of a matrix is equal to the number of nonzero singular values. The largest singular value σ1 is the solution to the optimization problem : ||Ax||2 σ1 = max x=0 ||x||2 It can be veried that the largest singular value satises the properties of a norm, it is called the spectral norm of the matrix. CS221 Linear Algebra Review 10/11/2011 34 / 37
  • 146. Matrix DecompositionsProperties of the Singular Value Decomposition The eigenvalues of the symmetric matrix AT A are equal to the square of the singular values of A : AT A = VΣUT UΣVT = VΣ2 VT The rank of a matrix is equal to the number of nonzero singular values. The largest singular value σ1 is the solution to the optimization problem : ||Ax||2 σ1 = max x=0 ||x||2 It can be veried that the largest singular value satises the properties of a norm, it is called the spectral norm of the matrix. In statistics analyzing data with the singular value decomposition is called Principal Component Analysis. CS221 Linear Algebra Review 10/11/2011 34 / 37
  • 147. Application : Image Compression1 Vectors2 Matrices3 Matrix Decompositions4 Application : Image Compression CS221 Linear Algebra Review 10/11/2011 35 / 37
  • 148. Application : Image CompressionCompression using the Singular Value Decomposition The singular value decomposition can be viewed as a sum of rank 1 matrices : CS221 Linear Algebra Review 10/11/2011 36 / 37
  • 149. Application : Image CompressionCompression using the Singular Value Decomposition The singular value decomposition can be viewed as a sum of rank 1 matrices : If some of the singular values are very small, we can discard them. CS221 Linear Algebra Review 10/11/2011 36 / 37
  • 150. Application : Image CompressionCompression using the Singular Value Decomposition The singular value decomposition can be viewed as a sum of rank 1 matrices : If some of the singular values are very small, we can discard them. This is a form of lossy compression. CS221 Linear Algebra Review 10/11/2011 36 / 37
  • 151. Application : Image CompressionCompression using the Singular Value Decomposition Truncating the singular value decomposition allows us to represent the matrix with less parameters. CS221 Linear Algebra Review 10/11/2011 37 / 37
  • 152. Application : Image CompressionCompression using the Singular Value Decomposition Truncating the singular value decomposition allows us to represent the matrix with less parameters. For a 512 × 512 matrix : CS221 Linear Algebra Review 10/11/2011 37 / 37
  • 153. Application : Image CompressionCompression using the Singular Value Decomposition Truncating the singular value decomposition allows us to represent the matrix with less parameters. For a 512 × 512 matrix : Full representation : 512 × 512 =262 144 CS221 Linear Algebra Review 10/11/2011 37 / 37
  • 154. Application : Image CompressionCompression using the Singular Value Decomposition Truncating the singular value decomposition allows us to represent the matrix with less parameters. For a 512 × 512 matrix : Full representation : 512 × 512 =262 144 Rank 10 approximation : 512 × 10 + 10 + 512 × 10 =10 250 CS221 Linear Algebra Review 10/11/2011 37 / 37
  • 155. Application : Image CompressionCompression using the Singular Value Decomposition Truncating the singular value decomposition allows us to represent the matrix with less parameters. For a 512 × 512 matrix : Full representation : 512 × 512 =262 144 Rank 10 approximation : 512 × 10 + 10 + 512 × 10 =10 250 Rank 40 approximation : 512 × 40 + 40 + 512 × 40 =41 000 CS221 Linear Algebra Review 10/11/2011 37 / 37
  • 156. Application : Image CompressionCompression using the Singular Value Decomposition Truncating the singular value decomposition allows us to represent the matrix with less parameters. For a 512 × 512 matrix : Full representation : 512 × 512 =262 144 Rank 10 approximation : 512 × 10 + 10 + 512 × 10 =10 250 Rank 40 approximation : 512 × 40 + 40 + 512 × 40 =41 000 Rank 80 approximation : 512 × 80 + 80 + 512 × 80 =82 000 CS221 Linear Algebra Review 10/11/2011 37 / 37