Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Successfully reported this slideshow.

Like this presentation? Why not share!

- What to Upload to SlideShare by SlideShare 5834569 views
- Customer Code: Creating a Company C... by HubSpot 4337641 views
- Be A Great Product Leader (Amplify,... by Adam Nash 973087 views
- Trillion Dollar Coach Book (Bill Ca... by Eric Schmidt 1145767 views
- APIdays Paris 2019 - Innovation @ s... by apidays 1321913 views
- A few thoughts on work life-balance by Wim Vanderbauwhede 1000640 views

824 views

Published on

Here is the link of the first part which was discussing linear systems: https://www.slideshare.net/CeniBabaogluPhDinMat/linear-algebra-for-machine-learning-linear-systems/1

Here are the slides of the second part which was discussing basis and dimension:

https://www.slideshare.net/CeniBabaogluPhDinMat/2-linear-algebra-for-machine-learning-basis-and-dimension

Here are the slides of the third part which is discussing factorization and linear transformations.

https://www.slideshare.net/CeniBabaogluPhDinMat/3-linear-algebra-for-machine-learning-factorization-and-linear-transformations-130813437

Published in:
Education

No Downloads

Total views

824

On SlideShare

0

From Embeds

0

Number of Embeds

205

Shares

0

Downloads

57

Comments

1

Likes

2

No notes for slide

- 1. Seminar Series on Linear Algebra for Machine Learning Part 4: Eigenvalues, Eigenvectors and Diagonalization Dr. Ceni Babaoglu Data Science Laboratory Ryerson University cenibabaoglu.com Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Eigenvalues, Eigenvectors and Diagonalization
- 2. Overview 1 Eigenvalues and eigenvectors 2 Some properties of eigenvalues and eigenvectors 3 Similar matrices 4 Diagonalizable matrices 5 Some properties of diagonalizable matrices 6 References Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Eigenvalues, Eigenvectors and Diagonalization
- 3. Eigenvalues and Eigenvectors When a matrix multiplies a vector in general the magnitude and direction of the vector will change. There are special vectors where only their magnitude is changed when multiplied by a matrix. These special vectors are called eigenvectors. The value by which the length changes is the associated eigenvalue. We say that x is an eigenvector of A if Ax = λx. λ is called the associated eigenvalue. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Eigenvalues, Eigenvectors and Diagonalization
- 4. Eigenvalues and Eigenvectors The matrix A transforms 5 diﬀerent vectors to other 5 diﬀerent vectors. Vector (1) is transformed to vector (a), vector (2) is transformed to vector (b) and so on. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Eigenvalues, Eigenvectors and Diagonalization
- 5. Eigenvalues and Eigenvectors All the vectors except vector (4) change both their magnitude and direction when transformed by A. Vector (4) changes only magnitude and does not change direction. Vector (4) is an eigenvector of A. λ = Magnitude of vector (d) Magnitude of vector (4) Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Eigenvalues, Eigenvectors and Diagonalization
- 6. Eigenvalues and Eigenvectors In this mapping the red arrow changes direction but the blue arrow does not. The blue arrow is an eigenvector of this mapping because it does not change direction, and since its length is unchanged, its eigenvalue is 1. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Eigenvalues, Eigenvectors and Diagonalization
- 7. Some properties If λ1, λ2, · · · , λn are distinct eigenvalues of a matrix, then the corresponding eigenvectors e1, e2, · · · , en are linearly independent. If e1 is an eigenvector of a matrix with corresponding eigenvalue λ1, then any nonzero scalar multiple of e1 is also an eigenvector with eigenvalue λ1. A real, symmetric square matrix has real eigenvalues, with orthogonal eigenvectors (can be chosen to be orthonormal). Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Eigenvalues, Eigenvectors and Diagonalization
- 8. Some properties The equation Ax = λx can be written in the form (A − λI)x = 0 (1) λ is an eigenvalue of A if and only if (1) has a nontrivial solution. (1) has nontrivial solution if and only if A − λI is singular, or equivalently det(A − λI) = 0 → characteristic equation for A (2) If (2) is expanded, we obtain an nth degree polynomial in λ, p(λ) = det(A − λI) → characteristic polynomial Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Eigenvalues, Eigenvectors and Diagonalization
- 9. Similar Matrices A matrix B is said to be similar to a matrix A if there exists a nonsingular matrix S such that B = S−1 AS. For n × n matrices A and B, if A is similar to B, then the two matrices both have the same characteristic polynomial and, consequently, both have the same eigenvalues. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Eigenvalues, Eigenvectors and Diagonalization
- 10. Diagonalizable Matrices An n × n matrix A is said to be diagonalizable if there exists a nonsingular matrix X and a diagonal matrix D such that X−1 AX = D. We say that X diagonalizes A. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Eigenvalues, Eigenvectors and Diagonalization
- 11. Some Properties An n × n matrix A is diagonalizable if and only if A has n linearly independent eigenvectors. If A is diagonalizable, then the column vectors of the diagonalizing matrix X are eigenvectors of A and the diagonal elements of D are the corresponding eigenvalues of A. The diagonalizing matrix X is not unique. Reordering the columns of a given diagonalizing matrix X or multiplying them by nonzero scalars will produce a new diagonalizing matrix. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Eigenvalues, Eigenvectors and Diagonalization
- 12. Some Properties If an n × n matrix A has n distinct eigenvalues, then A is diagonalizable. If the eigenvalues are not distinct, then A may or may not be diagonalizable, depending on whether A has n linearly independent eigenvectors. If A is diagonalizable, then A can be factored into a product XDX−1 . Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Eigenvalues, Eigenvectors and Diagonalization
- 13. Some Properties All the roots of the characteristic polynomial of a symmetric matrix are real numbers. If A is a symmetric matrix, then eigenvectors that belong to distinct eigenvalues of A are orthogonal. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Eigenvalues, Eigenvectors and Diagonalization
- 14. Example Diagonalize the matrix A = 2 −2 3 1 1 1 1 3 −1 We should solve the equation |A − λI| = 0. |A − λI| = 2 − λ −2 3 1 1 − λ 1 1 3 −1 − λ R2−R3→R2 −−−−−−−→ 2 − λ −2 3 0 −2 − λ 2 + λ 1 3 −1 − λ = (2 + λ) 2 − λ −2 3 0 −1 1 1 3 −1 − λ C2+C3→C2 −−−−−−−→ (2 + λ) 2 − λ 1 3 0 0 1 1 2 − λ −1 − λ = −(2 + λ) 2 − λ 1 1 2 − λ = −(2 + λ)[(2 − λ)2 − 1] = −(2 + λ)(λ2 − 4λ + 3) = 0. Eigenvalues: λ1 = 1, λ2 = 3, λ3 = −2. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Eigenvalues, Eigenvectors and Diagonalization
- 15. Example A = 2 −2 3 1 1 1 1 3 −1 We solve the equation |A − λI| = 0. Eigenvalues: λ1 = 1, λ2 = 3, λ3 = −2. The 3 × 3 matrix A has 3 distinct real eigenvalues. It has 3 linearly independent eigenvectors which implies A is diagonalizable. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Eigenvalues, Eigenvectors and Diagonalization
- 16. Example To ﬁnd the eigenvector corresponding to λ1 = 1, we solve the homogeneous system (A − λ1I) u = 0. (A − I) u = 0 ⇒ 1 −2 3 1 0 1 1 3 −2 · u1 u2 u3 = 0 ⇒ 1 −2 3 1 0 1 1 3 −2 R2−R1→R2 R3−R1→R3 −−−−−−−→ 1 −2 3 0 2 −2 0 5 −5 R2/2→R2 R3/5→R3 −−−−−→ 1 −2 3 0 1 −1 0 1 −1 R1+2R2→R1 R3−R2→R3 −−−−−−−→ 1 0 1 0 1 −1 0 0 0 ⇒ u1 + u3 = 0 u2 − u3 = 0 ⇒ u1 = −u2 = −u3 = α An eigenvector corresponding to λ1 = 1 is −1 1 1 Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Eigenvalues, Eigenvectors and Diagonalization
- 17. Example To ﬁnd the eigenvector corresponding to λ2 = 3, we solve the homogeneous system (A − λ2I) v = 0. ⇒ (A − 3I) v = 0 ⇒ −1 −2 3 1 −2 1 1 3 −4 · v1 v2 v3 = 0 ⇒ −1 −2 3 1 −2 1 1 3 −4 R2+R1→R2 R3+R1→R3 −−−−−−−→ −1 −2 3 0 −4 4 0 1 −1 −R1→R1 −R2/4→R2 −−−−−−−→ 1 2 −3 0 1 −1 0 1 −1 R1−2R2→R1 R3−R2→R3 −−−−−−−→ 1 0 −1 0 1 −1 0 0 0 ⇒ v1 − v3 = 0 v2 − v3 = 0 ⇒ v1 = v2 = v3 = α An eigenvector corresponding to λ2 = 3 is 1 1 1 Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Eigenvalues, Eigenvectors and Diagonalization
- 18. Example To ﬁnd the eigenspace corresponding to λ3 = −2, we solve the homogeneous system (A − λ3I) w = 0. ⇒ (A + 2I) w = 0 ⇒ 4 −2 3 1 3 1 1 3 1 · w1 w2 w3 = 0 ⇒ 4 −2 3 1 3 1 1 3 1 R1−4R2→R1 R3−R2→R3 −−−−−−−→ 0 −14 −1 1 3 1 0 0 0 R1↔R2 −−−−→ 1 3 1 0 −14 −1 0 0 0 −R2/14→R2 −−−−−−−→ 1 3 1 0 1 1/14 0 0 0 R1−3R2→R1 −−−−−−−→ 1 0 11/14 0 1 1/14 0 0 0 ⇒ w1 + 11w3/14 = 0 w2 + w3/14 = 0 ⇒ w1 = 11α w2 = α w3 = −14α An eigenvector corresponding to λ3 = −2 is 11 1 −14 Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Eigenvalues, Eigenvectors and Diagonalization
- 19. Example X−1 AX = D −1/2 5/6 −1/3 1/2 1/10 2/5 0 1/15 −1/15 2 −2 3 1 1 1 1 3 −1 −1 1 11 1 1 1 1 1 −14 = 1 0 0 0 3 0 0 0 −2 A = XDX−1 2 −2 3 1 1 1 1 3 −1 = −1 1 11 1 1 1 1 1 −14 1 0 0 0 3 0 0 0 −2 −1/2 5/6 −1/3 1/2 1/10 2/5 0 1/15 −1/15 Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Eigenvalues, Eigenvectors and Diagonalization
- 20. References Linear Algebra With Applications, 7th Edition by Steven J. Leon. Elementary Linear Algebra with Applications, 9th Edition by Bernard Kolman and David Hill. http://www.sharetechnote.com/html/Handbook_EngMath_Matrix_ Eigen.html https://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Eigenvalues, Eigenvectors and Diagonalization

No public clipboards found for this slide

Login to see the comments