# 2. Linear Algebra for Machine Learning: Basis and Dimension

C
1 of 20

## Recommended

Independence, basis and dimension by
Independence, basis and dimensionATUL KUMAR YADAV
4K views21 slides
3. Linear Algebra for Machine Learning: Factorization and Linear Transformations by
3. Linear Algebra for Machine Learning: Factorization and Linear TransformationsCeni Babaoglu, PhD
1.7K views27 slides
Lesson02 Vectors And Matrices Slides by
Lesson02 Vectors And Matrices SlidesMatthew Leingang
1.1K views65 slides
5. Linear Algebra for Machine Learning: Singular Value Decomposition and Prin... by
5. Linear Algebra for Machine Learning: Singular Value Decomposition and Prin...Ceni Babaoglu, PhD
3.2K views24 slides
Eigenvalue problems .ppt by
Eigenvalue problems .pptSelf-employed
7.9K views18 slides
02 linear algebra by
02 linear algebraRonald Teo
1.7K views23 slides

## What's hot

Svd by
Svdssuser1163cd
250 views33 slides
Eigen values and eigen vectors by
Eigen values and eigen vectorsRiddhi Patel
5.8K views14 slides
Linear Algebra and Matrix by
Linear Algebra and Matrixitutor
27.3K views31 slides
Eigen values and eigenvectors by
Eigen values and eigenvectorsAmit Singh
5K views50 slides
My Lecture Notes from Linear Algebra by
My Lecture Notes from Linear AlgebraPaul R. Martin
3.9K views99 slides
Numerical Methods - Power Method for Eigen values by
Numerical Methods - Power Method for Eigen valuesDr. Nirav Vyas
20.5K views37 slides

### What's hot(20)

Eigen values and eigen vectors by Riddhi Patel
Eigen values and eigen vectors
Riddhi Patel5.8K views
Linear Algebra and Matrix by itutor
Linear Algebra and Matrix
itutor27.3K views
Eigen values and eigenvectors by Amit Singh
Eigen values and eigenvectors
Amit Singh5K views
My Lecture Notes from Linear Algebra by Paul R. Martin
My Lecture Notes from Linear Algebra
Paul R. Martin3.9K views
Numerical Methods - Power Method for Eigen values by Dr. Nirav Vyas
Numerical Methods - Power Method for Eigen values
Dr. Nirav Vyas20.5K views
Eigen value and eigen vector by Rutvij Patel
Eigen value and eigen vector
Rutvij Patel9.3K views
ppt on Vector spaces (VCLA) by dhrumil patel and harshid panchal by harshid panchal
ppt on Vector spaces (VCLA) by dhrumil patel and harshid panchal
harshid panchal12.4K views
Complex Number's Applications by Nikhil Deswal
Complex Number's Applications
Nikhil Deswal21.1K views
systems of linear equations & matrices by Student
systems of linear equations & matrices
Student8.8K views
Linear Algebra and Matlab tutorial by Jia-Bin Huang
Linear Algebra and Matlab tutorial
Jia-Bin Huang5.5K views
Eigenvalues and Eigenvector by Muhammad Hamza
Eigenvalues and Eigenvector
Maths-->>Eigenvalues and eigenvectors by Jaydev Kishnani
Maths-->>Eigenvalues and eigenvectors
Jaydev Kishnani15.2K views
Chapter 1: First-Order Ordinary Differential Equations/Slides by Chaimae Baroudi
Chapter 1: First-Order Ordinary Differential Equations/Slides
Chaimae Baroudi1.1K views
Applications of numerical methods by Tarun Gehlot
Applications of numerical methods
Tarun Gehlot36.8K views
Interpolation with unequal interval by Dr. Nirav Vyas
Interpolation with unequal interval
Dr. Nirav Vyas38.5K views

## Similar to 2. Linear Algebra for Machine Learning: Basis and Dimension

Math for Intelligent Systems - 01 Linear Algebra 01 Vector Spaces by
Math for Intelligent Systems - 01 Linear Algebra 01 Vector SpacesAndres Mendez-Vazquez
687 views16 slides
Chapter 4: Vector Spaces - Part 1/Slides By Pearson by
Chapter 4: Vector Spaces - Part 1/Slides By PearsonChaimae Baroudi
1.3K views28 slides
Vectors and Matrices: basis and dimension by
Vectors and Matrices: basis and dimensionMarry Chriselle Rañola
536 views3 slides
Chapter 12 Section 12.1 Three-Dimensional Coordinate Sys by
Chapter 12 Section 12.1 Three-Dimensional Coordinate SysEstelaJeffery653
4 views34 slides
_lecture_02_cross_products_lines_and_planes.pdf by
_lecture_02_cross_products_lines_and_planes.pdfLeoIrsi
13 views42 slides

### Similar to 2. Linear Algebra for Machine Learning: Basis and Dimension(20)

Math for Intelligent Systems - 01 Linear Algebra 01 Vector Spaces by Andres Mendez-Vazquez
Math for Intelligent Systems - 01 Linear Algebra 01 Vector Spaces
Chapter 4: Vector Spaces - Part 1/Slides By Pearson by Chaimae Baroudi
Chapter 4: Vector Spaces - Part 1/Slides By Pearson
Chaimae Baroudi1.3K views
Chapter 12 Section 12.1 Three-Dimensional Coordinate Sys by EstelaJeffery653
Chapter 12 Section 12.1 Three-Dimensional Coordinate Sys
_lecture_02_cross_products_lines_and_planes.pdf by LeoIrsi
_lecture_02_cross_products_lines_and_planes.pdf
LeoIrsi13 views
Beginning direct3d gameprogrammingmath03_vectors_20160328_jintaeks by JinTaek Seo
Beginning direct3d gameprogrammingmath03_vectors_20160328_jintaeks
JinTaek Seo193 views
Mathematics for Deep Learning (1) by Ryoungwoo Jang
Mathematics for Deep Learning (1)
Ryoungwoo Jang164 views
2016 - Nonlinear Eigenvalue Problems And Contour Integrals by Dereck Downing
2016 - Nonlinear Eigenvalue Problems And Contour Integrals
Dereck Downing19 views
ALA Solution.pdf by RkAA4
ALA Solution.pdf
RkAA436 views
APLICACIONES DE ESPACIOS Y SUBESPACIOS VECTORIALES EN LA CARRERA DE ELECTRÓNI... by GersonMendoza15
APLICACIONES DE ESPACIOS Y SUBESPACIOS VECTORIALES EN LA CARRERA DE ELECTRÓNI...
GersonMendoza1597 views
M01L01 Advance Engineering Mathematics.pptx by SaurabhKalita5
SaurabhKalita54 views
Vector Algebra.pptx by azrulZamir2
Vector Algebra.pptx
azrulZamir286 views
Analytical Geometry in three dimension by SwathiSundari
Analytical Geometry in three dimension
SwathiSundari1.1K views

Career Building in AI - Technologies, Trends and Opportunities by
Career Building in AI - Technologies, Trends and OpportunitiesWebStackAcademy
51 views44 slides
BUSINESS ETHICS MODULE 1 UNIT I_B.pdf by
BUSINESS ETHICS MODULE 1 UNIT I_B.pdfDr Vijay Vishwakarma
55 views21 slides
Guess Papers ADC 1, Karachi University by
Guess Papers ADC 1, Karachi UniversityKhalid Aziz
109 views17 slides
Guidelines & Identification of Early Sepsis DR. NN CHAVAN 02122023.pptx by
Guidelines & Identification of Early Sepsis DR. NN CHAVAN 02122023.pptxNiranjan Chavan
43 views48 slides
Education of marginalized and socially disadvantages segments.pptx by
Education of marginalized and socially disadvantages segments.pptxGarimaBhati5
52 views36 slides
OOPs - JAVA Quick Reference.pdf by
OOPs - JAVA Quick Reference.pdfArthyR3
76 views66 slides

Career Building in AI - Technologies, Trends and Opportunities by WebStackAcademy
Career Building in AI - Technologies, Trends and Opportunities
Guess Papers ADC 1, Karachi University by Khalid Aziz
Guess Papers ADC 1, Karachi University
Khalid Aziz109 views
Guidelines & Identification of Early Sepsis DR. NN CHAVAN 02122023.pptx by Niranjan Chavan
Guidelines & Identification of Early Sepsis DR. NN CHAVAN 02122023.pptx
Niranjan Chavan43 views
Education of marginalized and socially disadvantages segments.pptx by GarimaBhati5
Education of marginalized and socially disadvantages segments.pptx
GarimaBhati552 views
OOPs - JAVA Quick Reference.pdf by ArthyR3
OOPs - JAVA Quick Reference.pdf
ArthyR376 views
Presentation_NC_Future now 2006.pdf by Lora
Presentation_NC_Future now 2006.pdf
Lora 38 views
Peripheral artery diseases by Dr. Garvit.pptx by garvitnanecha
Peripheral artery diseases by Dr. Garvit.pptx
garvitnanecha135 views
11.21.23 Economic Precarity and Global Economic Forces.pptx by mary850239
11.21.23 Economic Precarity and Global Economic Forces.pptx
mary85023994 views
Payment Integration using Braintree Connector | MuleSoft Mysore Meetup #37 by MysoreMuleSoftMeetup
Payment Integration using Braintree Connector | MuleSoft Mysore Meetup #37
BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (FRIE... by Nguyen Thanh Tu Collection
BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (FRIE...
INT-244 Topic 6b Confucianism by S Meyer
INT-244 Topic 6b Confucianism
S Meyer51 views
UNIT NO 13 ORGANISMS AND POPULATION.pptx by Madhuri Bhande
UNIT NO 13 ORGANISMS AND POPULATION.pptx
Creative Restart 2023: Christophe Wechsler - From the Inside Out: Cultivating... by Taste
Creative Restart 2023: Christophe Wechsler - From the Inside Out: Cultivating...
Taste39 views

### 2. Linear Algebra for Machine Learning: Basis and Dimension

• 1. Seminar Series on Linear Algebra for Machine Learning Part 2: Basis and Dimension Dr. Ceni Babaoglu Ryerson University cenibabaoglu.com Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Basis and Dimension
• 2. Overview 1 The span of a set of vectors 2 Linear dependence and independence 3 Basis and Dimension 4 Change of Basis 5 Changing Coordinates 6 Orthogonal and Orthonormal Bases 7 References Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Basis and Dimension
• 3. Vector Space A real vector space is a set V of elements on which we have two operations: ⊕ and deﬁned with the following properties. If x and y are any elements in V , then x ⊕ y ∈ V . (V is closed under ⊕) A1. x ⊕ y = y ⊕ x for x, y ∈ V . A2. (x ⊕ y) ⊕ z = x ⊕ (y ⊕ z) for x, y, z ∈ V . A3. There exists an element 0 in V such that x ⊕ 0 = x for x ∈ V . A4. For x ∈ V , there exists an element −x such that x ⊕ (−x) = 0. If x is any element in V and α is any real number, then α ⊕ x ∈ V . (V is closed under ) A5. α (x ⊕ y) = α x ⊕ α y for α ∈ R and x, y ∈ V . A6. (α + β) x = α x ⊕ β x for α, β ∈ R and x, y ∈ V . A7. (α β) x = α (β x) for α, β ∈ R and x, y ∈ V . A8. 1 x = x for x ∈ V . Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Basis and Dimension
• 4. The span of a set of vectors The set of all linear combinations of a set of vectors {v1, v2, · · · , vn} is called the span. Let S = {v1, v2, · · · , vn} be a subset of a vector space V . We say S spans V , if every vector in V can be written as a linear combination of vectors in S: v = c1v1 + c2v2 + · · · + cnvn. Note: A linear combination is in the form c1v1 + c2v2 + · · · + cnvn where ci ’s are real numbers. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Basis and Dimension
• 5. Linear dependence and independence The vectors {v1, v2, · · · , vn}. are linearly dependent if there exist scalars, ci , i = 1, 2, · · · , n, not all zero, such that c1v1 + c2v2 + · · · + cnvn = 0. The vectors {v1, v2, · · · , vn} are linearly independent if and only if ci = 0, i = 1, 2, · · · , n, is the only solution to c1v1 + c2v2 + · · · + cnvn = 0. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Basis and Dimension
• 6. Basis The vectors v1, v2, · · · , vn form a basis for a vector space V if and only if (i) v1, v2, · · · , vn are linearly independent. (ii) v1, v2, · · · , vn span V. If {v1, v2, · · · , vn} is a spanning set for a vector space V , then any collection of m vectors in V, where m > n, is linearly dependent. If {v1, v2, · · · , vn} and {u1, u2, · · · , um} are both bases for a vector space V , then n = m. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Basis and Dimension
• 7. Dimension If a vector space V has a basis consisting of n vectors, we say that V has dimension n. If V is a vector space of dimension n, then (I) Any set of n linearly independent vectors spans V . (II) Any n vectors that span V are linearly independent. (III) No set of less than n vectors can span V . (IV) Any subset of less than n linearly independent vectors can be extended to form a basis for V . (V) Any spanning set containing more than n vectors can be pared down to form a basis for V . Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Basis and Dimension
• 8. Example Rn is n-dimensional. Let x = [x1 x2 x3 · · · xn]T ∈ Rn. Then      x1 x2 ... xn      = x1        1 0 0 ... 0        + x2        0 1 0 ... 0        + · · · + xn        0 0 0 ... 1        ≡ x1i1 + x2i2 + · · · + xnin. Therefore span{i1, i2, · · · , in} = Rn. i1, i2, · · · , in are linearly independent. They form a basis and this implies the dimension of R is n. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Basis and Dimension
• 9. Note The basis in the previous example is the standard basis for Rn. In R3 the following notation is used for the standard basis: i =   1 0 0   , j =   0 1 0   , k =   0 0 1   . In an n-dimensional vector space V , set of n elements that span V must be independent and any set of n independent elements must span V . Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Basis and Dimension
• 10. Change of Basis The standard basis for R2 is {e1, e2}. Any vector x ∈ R2 can be expressed as a linear combination x = x1e1 + x2e2. The scalars x1 and x2 can be thought of as the coordinates of x with respect to the standard basis. For any basis {y, z} for R2, a given vector x can be represented uniquely as a linear combination, x = αy + βz. [y, z]: ordered basis (α, β)T : the coordinate vector of x with respect to [y, z] If we reverse the order of the basis vectors and take [z, y], then we must also reorder the coordinate vector and take (β, α)T . Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Basis and Dimension
• 11. Example y = (2, 1)T and z = (1, 4)T are linearly independent and form a basis for R2. x = (7, 7)T can be written as a linear combination: x = 3y + z The coordinate vector of x with respect to [y, z] is (3, 1)T . Geometrically, the coordinate vector speciﬁes how to get from the origin O(0, 0) to the point P(7, 7), moving ﬁrst in the direction of y and then in the direction of z . Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Basis and Dimension
• 12. Changing Coordinates Suppose, for example, instead of using [e1, e2] for R2, we wish to use a diﬀerent basis, say u1 = 3 2 , u2 = 1 1 . I. Given a vector c1u1 + c2u2, let’s ﬁnd its coordinates with respect to e1 and e2. II. Given a vector x = (x1, x2)T , let’s ﬁnd its coordinates with respect to u1 and u2. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Basis and Dimension
• 13. Changing Coordinates I. Given a vector c1u1 + c2u2, let’s ﬁnd its coordinates with respect to e1 and e2. u1 = 3e1 + 2e2, u2 = e1 + e2 c1u1 + c2u2 = (3c1e1 + 2c1e2) + (c2e1 + c2e2) = (3c1 + c2)e1 + (2c1 + c2)e2 x = 3c1 + c2 2c1 + c2 = 3 1 2 1 c1 c2 U = (u1, u2) = 3 1 2 1 U: the transition matrix from [u1, u2] to [e1, e2] Given any coordinate c with respect to [u1, u2], the corresponding coordinate vector x with respect to [e1, e2] by x = Uc Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Basis and Dimension
• 14. Changing Coordinates II. Given a vector x = (x1, x2)T , let’s ﬁnd its coordinates with respect to u1 and u2. We have to ﬁnd the transition matrix from [e1, e2] to [u1, u2]. The matrix U is nonsingular, since its column vectors u1 and u2 are linearly independent. c = U−1 x Given vector x, x = (x1, x2)T = x1e1 + x2e2 We need to multiply by U−1 to ﬁnd its coordinate vector with respect to [u1, u2]. U−1: the transition matrix from [e1, e2] to [u1, u2] Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Basis and Dimension
• 15. Example Let E = (1, 2)T , (0, 1)T and F = (1, 3)T , (−1, 2)T be ordered basis for R2. (i) Find the transition matrix S from the basis E to the basis F. (ii) If the vector x ∈ R2 has the coordinate vector [x]E = (5, 0)T with respect to the ordered basis E, determine the coordinate vector [x]F with respect to the ordered basis F. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Basis and Dimension
• 16. Example (i) We should ﬁnd the matrix S = a c b d such that [x]F = S [x]E for x ∈ R2. We solve the following system: 1 2 = a 1 3 + b −1 2 0 1 = c 1 3 + d −1 2 ⇒ 1 = a − b 2 = 3a + 2b 0 = c − d 1 = 3c + 2d ⇒ a = 4/5 b = −1/5 c = d = 1/5 ⇒ S = 4/5 1/5 −1/5 1/5 Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Basis and Dimension
• 17. Example E -V (1, 0)T , (0, 1)T ? U−1 Z Z Z Z Z Z Z ZZ~ U−1V F where V = 1 0 2 1 and U = 1 −1 3 2 . Thus we obtain U−1 = 2/5 1/5 −3/5 1/5 and then S = U−1 V = 4/5 1/5 −1/5 1/5 . Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Basis and Dimension
• 18. Example (ii) Since [x]E = (5, 0)T , [x]F = S [x]E = 4/5 1/5 −1/5 1/5 5 0 = 4 −1 Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Basis and Dimension
• 19. Orthogonal and Orthonormal Bases n linearly independent real vectors span Rn and they form a basis for the space. An orthogonal basis, a1, · · · , an satisﬁes ai · aj = 0, if i = j An orthonormal basis, a1, · · · , an satisﬁes ai · aj = 0, if i = j ai · aj = 1, if i = j Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Basis and Dimension
• 20. References Linear Algebra With Applications, 7th Edition by Steven J. Leon. Elementary Linear Algebra with Applications, 9th Edition by Bernard Kolman and David Hill. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Basis and Dimension
Current LanguageEnglish
Español
Portugues
Français
Deutsche