SlideShare a Scribd company logo
1 of 27
Download to read offline
Seminar Series on
Linear Algebra for Machine Learning
Part 3: Factorization and Linear Transformations
Dr. Ceni Babaoglu
Data Science Laboratory
Ryerson University
cenibabaoglu.com
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Overview
1 Row and Column Spaces
2 Rank of a Matrix
3 Rank and Singularity
4 Inner Product Spaces
5 Gram-Schmidt Process
6 Factorization
7 Linear Transformation
8 Linear Transformation and Singularity
9 Similar Matrices
10 References
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Row and Column Spaces
Let
A =





a11 a12 a13 . . . a1n
a21 a22 a23 . . . a2n
...
...
...
...
...
am1 am2 am3 . . . amn





be an m × n matrix.
The rows of A, considered as vectors in Rn, span a subspace
of Rn called the row space of A.
Similarly, the columns of A, considered as vectors in Rm, span
a subspace of Rm called the column space of A.
If A and B are two m × n row (column) equivalent matrices,
then the row (column) spaces of A and B are equal.
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Rank of a Matrix
The dimension of the row (column) space of A is called the
row (column) rank of A.
The row rank and column rank of the m × n matrix A = [aij ]
are equal.
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Rank and Singularity
A is nonsingular.
Ax = 0 has only the trivial solution.
A is row (column) equivalent to In.
For every vector b in Rn, the system Ax = b has a unique
solution.
det(A) = 0.
The rank of A is n.
The rows of A form a linearly independent set of vectors in Rn.
The columns of A form a linearly independent set of vectors in
Rn.
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Example
Let A =


1 −1 2 0 −3
0 1 0 4 0
2 −1 4 4 −6

. Find the following:
(i) A basis for the column space of A and its dimension.
(ii) A basis for the row space of A and its dimension.
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Example


1 −1 2 0 −3
0 1 0 4 0
2 −1 4 4 −6

 S3−2S1→S3
−−−−−−−→


1 −1 2 0 −3
0 1 0 4 0
0 1 0 4 0


S1+S2→S1
S3−S2→S3
−−−−−−→


1 0 2 4 −3
0 1 0 4 0
0 0 0 0 0


(i) The column space of A is spanned by the vectors (1, 0, 2)T
and
(−1, 1, −1)T
. These vectors are linearly independent.
{(1, 0, 2)T
, (−1, 1, −1)T
} is a basis for this space and its dimension is 2.
(ii) The row space of A is spanned by the vectors (1, 0, 2, 4, −3) and
(0, 1, 0, 4, 0). These vectors are linearly independent.
{(1, 0, 2, 4, −3), (0, 1, 0, 4, 0)} is a basis for this space and its
dimension is 2.
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Inner Product Spaces
Let V be a real vector space. An inner product on V is a function
that assigns to each ordered pair of vectors u, v in V real number
(u, v) satisfying the following properties:
(u, u) ≥ 0, (u, u) = 0 if and only if u = 0v
(v, u) = (u, v) for any u, v in V
(u + v, w) = (u, w) + (v, w) for any u, v, w in V
(cu, v) = c(u, v) for u, v in V and c, a real scalar
A real vector space that has an inner product defined on it is
called an inner product space. If the space is finite
dimensional it is called a Euclidean space.
In an inner product space we define the length of
a vector u by u = (u, u).
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Gram-Schmidt Process
Let V be an inner product space and W = {0} an m-dimensional
subspace of V . Then there exists an orthonormal basis
T = {w1, w2, · · · , wm} for W .
Let S = {u1, u2, · · · , um} be any basis for W . Construct an
orthogonal basis T∗ = {v1, v2, · · · , vm} for W. Select any one of
the vectors in S, say u1 and call it v1. Look for a vector v2 in the
subspace W1 of W spanned by {u1, u2} that is orthogonal to v1.
v2 = u2 −
(u2, v1)
(v1, v1)
v1
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Gram-Schmidt Process
Next, we look for a vector v3 in the subspace W2 of W spanned by
{u1, u2, u3} that is orthogonal to both v1 and v2.
v2 = u2 −
(u2, v1)
(v1, v1)
v1
v3 = u3 −
(u3, v1)
(v1, v1)
v1 −
(u3, v2)
(v2, v2)
v2
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Example
Let’s use the Gram-Schmidt process to find an orthonormal basis
for the subspace of R4 with basis u1 = (1, 1, 1, 0)T ,
u2 = (−1, 0, −1, 1)T and u3 = (−1, 0, 0, −1)T . First let v1 = u1,
v2 = u2 −
(u2, v1)
(v1, v1)
v1 =




−1
0
−1
1



 − (−
2
3
)




1
1
1
1



 =




−1/3
2/3
−1/3
1




Multiplying v2 by 3 to clear fractions, we get




−1
2
−1
3




Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Example
v3 = u3 −
(u3, v1)
(v1, v1)
v1 −
(u3, v2)
(v2, v2)
v2 =




−4/5
3/5
1/5
−3/5




Multiplying v3 by 5 to clear fractions, we get (−4, 3, 1, −3)T . An
orthogonal basis:
{v1, v2, v3} =




1
1
1
0



 ,




−1
2
−1
3



 ,




−4
3
1
−3




An orthonormal basis:
{w1, w2, w3} =




1/
√
3
1/
√
3
1/
√
3
0



 ,




−1/
√
15
2/
√
15
−1/
√
15
3/
√
15



 ,




−4/
√
35
3/
√
35
1/
√
35
−3/
√
35




Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Factorization
If A is an m × n matrix with linearly independent columns, then A
can be factored as
A = QR,
Q: an m × n matrix whose columns form an orthonormal basis for
the column space of A,
R: an n × n nonsingular upper triangular matrix.
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Example
Let’s find the factorization of




1 −1 −1
1 0 0
1 −1 0
0 1 −1



 .
Let’s define the columns of A as the vectors u1, u2, u3.
The orthonormal basis for the column space of A is
w1 =




1/
√
3
1/
√
3
1/
√
3
0



 , w2 =




−1/
√
15
2/
√
15
−1/
√
15
3/
√
15



 , w3 =




−4/
√
35
3/
√
35
1/
√
35
−3/
√
35



 .
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Example
Q =




1/
√
3 −1/
√
15 −4/
√
35
1/
√
3 2/
√
15 3/
√
35
1/
√
3 −1/
√
15 1/
√
35
0 3/
√
15 −3/
√
35




R =


r11 r12 r13
0 r22 r23
0 0 r33


where rji = (ui , wj ).
R =


3/
√
3 −2/
√
3 −1/
√
3
0 5/
√
15 −2/
√
15
0 0 7/
√
35


A = QR
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Linear Transformation
A mapping L : V −→ W is said to be a linear transformation or a
linear operator if
L(αv1 + βv2) = αL(v1) + βL(v2)
OR
L(v1 + v2) = L(v1) + L(v2), (α = β = 1)
L(αv) = αL(v) (v = v1, β = 0)
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Example
L(x) = 3x, x ∈ R2
.
L(x + y) = 3(x + y) = L(x + y)
L(αx) = 3(αx) = αL(x)
L is a linear transformation.
α : positive scalar
F(x) = αx can be thought of as a stretching or shrinking by a
factor of α.
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Example
L(x) = x1e1, x ∈ R2
.
If x = (x1, x2)T
, then L(x) = (x, 0)T
If y = (y1, y2)T
, then αx + βy =
αx1 + βy1
αx2 + βy2
L(αx + βy) = (αx1 + βy1)e1 = α(x1e1) + β(y1e1) = αL(x) + βL(y)
L is a linear transformation, a projection onto the x1 axis.
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Example
L(x) = (−x2, x1)T , x = (x1, x2)T ∈ R2.
L(αx + βy) =
−(αx2 + βy2)
αx1 + βy1
= α
−x2
x1
+ β
−y2
y1
= αL(x) + βL(y)
L is a linear transformation. It has the effect of rotating each vector in R2
by
90◦
in the counterclockwise direction.
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Example
M(x) = (x2
1 + x2
2 )1/2, M : R2 −→ R.
M(αx) = (α2
x2
1 + α2
x2
2 )1/2
=| α | M(x),
αM(x) = M(αx), α < 0, x = 0.
M is not a linear transformation.
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Linear One-to-one Transformations
A linear transformation L : V → W is called one-to-one if it is
a one-to-one function; that is, if v1 = v2 implies that
L(v1) = L(v2).
An equivalent statement is that L is one-to-one if
L(v1) = L(v2) implies that v1 = v2.
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Linear Onto Transformations
If L : V → W is a linear transformation of a vector space V
into a vector space W , then the range of L or image of V
under L, denoted by range L , consists of all those vectors in
W that are images under L of vectors in V .
Thus w is in range L if there exists some vector v in V such
that L(v) = w. The linear transformation L is called onto if
range L = W .
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Linear Transformation and Singularity
A is nonsingular.
Ax = 0 has only the trivial solution.
A is row (column) equivalent to In.
For every vector b in Rn, the system Ax = b has a unique
solution.
det(A) = 0.
The rank of A is n.
The rows of A form a linearly independent set of vectors in Rn.
The columns of A form a linearly independent set of vectors in
Rn.
The linear transformation L : Rn −→ Rn defined by
L(x) = A(x), for x in Rn, is one-to-one and onto.
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Similar Matrices
If A and B are n × n matrices, we say that B is similar to A if
there is a nonsingular matrix P such that B = P−1AP.
Let V be any n−dimensional vector space and let A and B be
any n × n matrices. Then A and B are similar if and only if A
and B represent the same linear transformation L : V → V
with respect to two ordered bases for V .
If A and B are similar n × n matrices, then rank A = rank B.
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Example
Let L : R3 → R3 be defined by
L([ u1 u2 u3 ]) = [ 2u1 − u3 u1 + u2 − u3 u3 ]
and S = {[1 0 0], [0 1 0], [0 0 1]} be the natural basis
for R3. The representation of L with respect of S is
A =


2 0 −1
1 1 −1
0 0 1

 .
Considering S = {[1 0 1], [0 1 0], [1 1 0]} as ordered
bases for R3, the transition matrix P from S to S is
P =


1 0 1
0 1 1
1 0 0

 P−1
=


0 0 1
−1 1 1
1 0 −1

 .
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
Example
Then the representation of L with respect to S is
B = P−1
AP =


1 0 0
0 1 0
0 0 2


The matrices
A =


2 0 −1
1 1 −1
0 0 1

 and B =


1 0 0
0 1 0
0 0 2


are similar.
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations
References
Linear Algebra With Applications, 7th Edition
by Steven J. Leon.
Elementary Linear Algebra with Applications, 9th Edition
by Bernard Kolman and David Hill.
Dr. Ceni Babaoglu cenibabaoglu.com
Linear Algebra for Machine Learning: Factorization and Linear Transformations

More Related Content

What's hot

Dimensionality Reduction
Dimensionality ReductionDimensionality Reduction
Dimensionality Reductionmrizwan969
 
Eigen values and eigen vectors engineering
Eigen values and eigen vectors engineeringEigen values and eigen vectors engineering
Eigen values and eigen vectors engineeringshubham211
 
Sequences and Series
Sequences and SeriesSequences and Series
Sequences and Seriessujathavvv
 
Machine Learning and Inductive Inference
Machine Learning and Inductive InferenceMachine Learning and Inductive Inference
Machine Learning and Inductive Inferencebutest
 
Eigen values and eigenvectors
Eigen values and eigenvectorsEigen values and eigenvectors
Eigen values and eigenvectorsAmit Singh
 
Fractional calculus and applications
Fractional calculus and applicationsFractional calculus and applications
Fractional calculus and applicationsPlusOrMinusZero
 
Inverse Matrix & Determinants
Inverse Matrix & DeterminantsInverse Matrix & Determinants
Inverse Matrix & Determinantsitutor
 
Support vector machine
Support vector machineSupport vector machine
Support vector machineMusa Hawamdah
 
QR Algorithm Presentation
QR Algorithm PresentationQR Algorithm Presentation
QR Algorithm Presentationkmwangi
 
Maths-->>Eigenvalues and eigenvectors
Maths-->>Eigenvalues and eigenvectorsMaths-->>Eigenvalues and eigenvectors
Maths-->>Eigenvalues and eigenvectorsJaydev Kishnani
 
Linear regression
Linear regressionLinear regression
Linear regressionMartinHogg9
 
Eigen value and eigen vectors shwetak
Eigen value and eigen vectors shwetakEigen value and eigen vectors shwetak
Eigen value and eigen vectors shwetakMrsShwetaBanait1
 
Integration and its basic rules and function.
Integration and its basic rules and function.Integration and its basic rules and function.
Integration and its basic rules and function.Kartikey Rohila
 
Support Vector Machine ppt presentation
Support Vector Machine ppt presentationSupport Vector Machine ppt presentation
Support Vector Machine ppt presentationAyanaRukasar
 
Introduction to Neural Networks
Introduction to Neural NetworksIntroduction to Neural Networks
Introduction to Neural NetworksDatabricks
 

What's hot (20)

Dimensionality Reduction
Dimensionality ReductionDimensionality Reduction
Dimensionality Reduction
 
Eigen values and eigen vectors engineering
Eigen values and eigen vectors engineeringEigen values and eigen vectors engineering
Eigen values and eigen vectors engineering
 
Linear Algebra
Linear AlgebraLinear Algebra
Linear Algebra
 
Sequences and Series
Sequences and SeriesSequences and Series
Sequences and Series
 
Machine Learning and Inductive Inference
Machine Learning and Inductive InferenceMachine Learning and Inductive Inference
Machine Learning and Inductive Inference
 
Eigen values and eigenvectors
Eigen values and eigenvectorsEigen values and eigenvectors
Eigen values and eigenvectors
 
Fractional calculus and applications
Fractional calculus and applicationsFractional calculus and applications
Fractional calculus and applications
 
Inverse Matrix & Determinants
Inverse Matrix & DeterminantsInverse Matrix & Determinants
Inverse Matrix & Determinants
 
Support vector machine
Support vector machineSupport vector machine
Support vector machine
 
Linear regression
Linear regressionLinear regression
Linear regression
 
QR Algorithm Presentation
QR Algorithm PresentationQR Algorithm Presentation
QR Algorithm Presentation
 
Maths-->>Eigenvalues and eigenvectors
Maths-->>Eigenvalues and eigenvectorsMaths-->>Eigenvalues and eigenvectors
Maths-->>Eigenvalues and eigenvectors
 
K means Clustering Algorithm
K means Clustering AlgorithmK means Clustering Algorithm
K means Clustering Algorithm
 
Ring
RingRing
Ring
 
Linear regression
Linear regressionLinear regression
Linear regression
 
Eigen value and eigen vectors shwetak
Eigen value and eigen vectors shwetakEigen value and eigen vectors shwetak
Eigen value and eigen vectors shwetak
 
Integration and its basic rules and function.
Integration and its basic rules and function.Integration and its basic rules and function.
Integration and its basic rules and function.
 
Numerical Method
Numerical Method Numerical Method
Numerical Method
 
Support Vector Machine ppt presentation
Support Vector Machine ppt presentationSupport Vector Machine ppt presentation
Support Vector Machine ppt presentation
 
Introduction to Neural Networks
Introduction to Neural NetworksIntroduction to Neural Networks
Introduction to Neural Networks
 

Similar to Linear Algebra Seminar on Factorization and Transformations

2. Linear Algebra for Machine Learning: Basis and Dimension
2. Linear Algebra for Machine Learning: Basis and Dimension2. Linear Algebra for Machine Learning: Basis and Dimension
2. Linear Algebra for Machine Learning: Basis and DimensionCeni Babaoglu, PhD
 
Proyecto grupal algebra parcial ii
Proyecto grupal algebra parcial iiProyecto grupal algebra parcial ii
Proyecto grupal algebra parcial iiJHANDRYALCIVARGUAJAL
 
Chapter 4: Vector Spaces - Part 1/Slides By Pearson
Chapter 4: Vector Spaces - Part 1/Slides By PearsonChapter 4: Vector Spaces - Part 1/Slides By Pearson
Chapter 4: Vector Spaces - Part 1/Slides By PearsonChaimae Baroudi
 
Math for Intelligent Systems - 01 Linear Algebra 01 Vector Spaces
Math for Intelligent Systems - 01 Linear Algebra 01  Vector SpacesMath for Intelligent Systems - 01 Linear Algebra 01  Vector Spaces
Math for Intelligent Systems - 01 Linear Algebra 01 Vector SpacesAndres Mendez-Vazquez
 
APLICACIONES DE ESPACIOS Y SUBESPACIOS VECTORIALES EN LA CARRERA DE ELECTRÓNI...
APLICACIONES DE ESPACIOS Y SUBESPACIOS VECTORIALES EN LA CARRERA DE ELECTRÓNI...APLICACIONES DE ESPACIOS Y SUBESPACIOS VECTORIALES EN LA CARRERA DE ELECTRÓNI...
APLICACIONES DE ESPACIOS Y SUBESPACIOS VECTORIALES EN LA CARRERA DE ELECTRÓNI...GersonMendoza15
 
Seismic data processing introductory lecture
Seismic data processing introductory lectureSeismic data processing introductory lecture
Seismic data processing introductory lectureAmin khalil
 
Vectorspace in 2,3and n space
Vectorspace in 2,3and n spaceVectorspace in 2,3and n space
Vectorspace in 2,3and n spaceAhmad Saifullah
 
Aplicaciones y subespacios y subespacios vectoriales en la
Aplicaciones y subespacios y subespacios vectoriales en laAplicaciones y subespacios y subespacios vectoriales en la
Aplicaciones y subespacios y subespacios vectoriales en laemojose107
 
Lecture2 (vectors and tensors).pdf
Lecture2 (vectors and tensors).pdfLecture2 (vectors and tensors).pdf
Lecture2 (vectors and tensors).pdfentesarkareem1
 
1625 signal processing and representation theory
1625 signal processing and representation theory1625 signal processing and representation theory
1625 signal processing and representation theoryDr Fereidoun Dejahang
 
Maths4ml linearalgebra-formula
Maths4ml linearalgebra-formulaMaths4ml linearalgebra-formula
Maths4ml linearalgebra-formulaNishant Upadhyay
 
Chapter 4: Vector Spaces - Part 4/Slides By Pearson
Chapter 4: Vector Spaces - Part 4/Slides By PearsonChapter 4: Vector Spaces - Part 4/Slides By Pearson
Chapter 4: Vector Spaces - Part 4/Slides By PearsonChaimae Baroudi
 
Notes on eigenvalues
Notes on eigenvaluesNotes on eigenvalues
Notes on eigenvaluesAmanSaeed11
 
Chapter 12 Section 12.1 Three-Dimensional Coordinate Sys
Chapter 12 Section 12.1  Three-Dimensional Coordinate SysChapter 12 Section 12.1  Three-Dimensional Coordinate Sys
Chapter 12 Section 12.1 Three-Dimensional Coordinate SysEstelaJeffery653
 
Orthogonal porjection in statistics
Orthogonal porjection in statisticsOrthogonal porjection in statistics
Orthogonal porjection in statisticsSahidul Islam
 

Similar to Linear Algebra Seminar on Factorization and Transformations (20)

2. Linear Algebra for Machine Learning: Basis and Dimension
2. Linear Algebra for Machine Learning: Basis and Dimension2. Linear Algebra for Machine Learning: Basis and Dimension
2. Linear Algebra for Machine Learning: Basis and Dimension
 
Proyecto grupal algebra parcial ii
Proyecto grupal algebra parcial iiProyecto grupal algebra parcial ii
Proyecto grupal algebra parcial ii
 
Chapter 4: Vector Spaces - Part 1/Slides By Pearson
Chapter 4: Vector Spaces - Part 1/Slides By PearsonChapter 4: Vector Spaces - Part 1/Slides By Pearson
Chapter 4: Vector Spaces - Part 1/Slides By Pearson
 
Math for Intelligent Systems - 01 Linear Algebra 01 Vector Spaces
Math for Intelligent Systems - 01 Linear Algebra 01  Vector SpacesMath for Intelligent Systems - 01 Linear Algebra 01  Vector Spaces
Math for Intelligent Systems - 01 Linear Algebra 01 Vector Spaces
 
APLICACIONES DE ESPACIOS Y SUBESPACIOS VECTORIALES EN LA CARRERA DE ELECTRÓNI...
APLICACIONES DE ESPACIOS Y SUBESPACIOS VECTORIALES EN LA CARRERA DE ELECTRÓNI...APLICACIONES DE ESPACIOS Y SUBESPACIOS VECTORIALES EN LA CARRERA DE ELECTRÓNI...
APLICACIONES DE ESPACIOS Y SUBESPACIOS VECTORIALES EN LA CARRERA DE ELECTRÓNI...
 
Seismic data processing introductory lecture
Seismic data processing introductory lectureSeismic data processing introductory lecture
Seismic data processing introductory lecture
 
Vectorspace in 2,3and n space
Vectorspace in 2,3and n spaceVectorspace in 2,3and n space
Vectorspace in 2,3and n space
 
Aplicaciones y subespacios y subespacios vectoriales en la
Aplicaciones y subespacios y subespacios vectoriales en laAplicaciones y subespacios y subespacios vectoriales en la
Aplicaciones y subespacios y subespacios vectoriales en la
 
Grupo 13 taller parcial2_nrc2882
Grupo 13 taller parcial2_nrc2882Grupo 13 taller parcial2_nrc2882
Grupo 13 taller parcial2_nrc2882
 
Lecture2 (vectors and tensors).pdf
Lecture2 (vectors and tensors).pdfLecture2 (vectors and tensors).pdf
Lecture2 (vectors and tensors).pdf
 
1625 signal processing and representation theory
1625 signal processing and representation theory1625 signal processing and representation theory
1625 signal processing and representation theory
 
Maths4ml linearalgebra-formula
Maths4ml linearalgebra-formulaMaths4ml linearalgebra-formula
Maths4ml linearalgebra-formula
 
1560 mathematics for economists
1560 mathematics for economists1560 mathematics for economists
1560 mathematics for economists
 
real vector space
real vector spacereal vector space
real vector space
 
Chapter 4: Vector Spaces - Part 4/Slides By Pearson
Chapter 4: Vector Spaces - Part 4/Slides By PearsonChapter 4: Vector Spaces - Part 4/Slides By Pearson
Chapter 4: Vector Spaces - Part 4/Slides By Pearson
 
Notes on eigenvalues
Notes on eigenvaluesNotes on eigenvalues
Notes on eigenvalues
 
Chapter 12 Section 12.1 Three-Dimensional Coordinate Sys
Chapter 12 Section 12.1  Three-Dimensional Coordinate SysChapter 12 Section 12.1  Three-Dimensional Coordinate Sys
Chapter 12 Section 12.1 Three-Dimensional Coordinate Sys
 
Orthogonal porjection in statistics
Orthogonal porjection in statisticsOrthogonal porjection in statistics
Orthogonal porjection in statistics
 
Vector Space.pptx
Vector Space.pptxVector Space.pptx
Vector Space.pptx
 
Vector space
Vector spaceVector space
Vector space
 

Recently uploaded

Scientific Writing :Research Discourse
Scientific  Writing :Research  DiscourseScientific  Writing :Research  Discourse
Scientific Writing :Research DiscourseAnita GoswamiGiri
 
physiotherapy in Acne condition.....pptx
physiotherapy in Acne condition.....pptxphysiotherapy in Acne condition.....pptx
physiotherapy in Acne condition.....pptxAneriPatwari
 
Congestive Cardiac Failure..presentation
Congestive Cardiac Failure..presentationCongestive Cardiac Failure..presentation
Congestive Cardiac Failure..presentationdeepaannamalai16
 
The role of Geography in climate education: science and active citizenship
The role of Geography in climate education: science and active citizenshipThe role of Geography in climate education: science and active citizenship
The role of Geography in climate education: science and active citizenshipKarl Donert
 
How to Uninstall a Module in Odoo 17 Using Command Line
How to Uninstall a Module in Odoo 17 Using Command LineHow to Uninstall a Module in Odoo 17 Using Command Line
How to Uninstall a Module in Odoo 17 Using Command LineCeline George
 
Sulphonamides, mechanisms and their uses
Sulphonamides, mechanisms and their usesSulphonamides, mechanisms and their uses
Sulphonamides, mechanisms and their usesVijayaLaxmi84
 
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...DhatriParmar
 
Geoffrey Chaucer Works II UGC NET JRF TGT PGT MA PHD Entrance Exam II History...
Geoffrey Chaucer Works II UGC NET JRF TGT PGT MA PHD Entrance Exam II History...Geoffrey Chaucer Works II UGC NET JRF TGT PGT MA PHD Entrance Exam II History...
Geoffrey Chaucer Works II UGC NET JRF TGT PGT MA PHD Entrance Exam II History...DrVipulVKapoor
 
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...DhatriParmar
 
Objectives n learning outcoms - MD 20240404.pptx
Objectives n learning outcoms - MD 20240404.pptxObjectives n learning outcoms - MD 20240404.pptx
Objectives n learning outcoms - MD 20240404.pptxMadhavi Dharankar
 
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 - I-LEARN SMART WORLD - CẢ NĂM - CÓ FILE NGHE (BẢN...
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 - I-LEARN SMART WORLD - CẢ NĂM - CÓ FILE NGHE (BẢN...BÀI TẬP BỔ TRỢ TIẾNG ANH 8 - I-LEARN SMART WORLD - CẢ NĂM - CÓ FILE NGHE (BẢN...
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 - I-LEARN SMART WORLD - CẢ NĂM - CÓ FILE NGHE (BẢN...Nguyen Thanh Tu Collection
 
BÀI TẬP BỔ TRỢ 4 KĨ NĂNG TIẾNG ANH LỚP 8 - CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC ...
BÀI TẬP BỔ TRỢ 4 KĨ NĂNG TIẾNG ANH LỚP 8 - CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC ...BÀI TẬP BỔ TRỢ 4 KĨ NĂNG TIẾNG ANH LỚP 8 - CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC ...
BÀI TẬP BỔ TRỢ 4 KĨ NĂNG TIẾNG ANH LỚP 8 - CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC ...Nguyen Thanh Tu Collection
 
PART 1 - CHAPTER 1 - CELL THE FUNDAMENTAL UNIT OF LIFE
PART 1 - CHAPTER 1 - CELL THE FUNDAMENTAL UNIT OF LIFEPART 1 - CHAPTER 1 - CELL THE FUNDAMENTAL UNIT OF LIFE
PART 1 - CHAPTER 1 - CELL THE FUNDAMENTAL UNIT OF LIFEMISSRITIMABIOLOGYEXP
 
Grade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptxGrade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptxkarenfajardo43
 
Narcotic and Non Narcotic Analgesic..pdf
Narcotic and Non Narcotic Analgesic..pdfNarcotic and Non Narcotic Analgesic..pdf
Narcotic and Non Narcotic Analgesic..pdfPrerana Jadhav
 
DBMSArchitecture_QueryProcessingandOptimization.pdf
DBMSArchitecture_QueryProcessingandOptimization.pdfDBMSArchitecture_QueryProcessingandOptimization.pdf
DBMSArchitecture_QueryProcessingandOptimization.pdfChristalin Nelson
 
Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...
Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...
Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...Osopher
 

Recently uploaded (20)

Scientific Writing :Research Discourse
Scientific  Writing :Research  DiscourseScientific  Writing :Research  Discourse
Scientific Writing :Research Discourse
 
CARNAVAL COM MAGIA E EUFORIA _
CARNAVAL COM MAGIA E EUFORIA            _CARNAVAL COM MAGIA E EUFORIA            _
CARNAVAL COM MAGIA E EUFORIA _
 
Introduction to Research ,Need for research, Need for design of Experiments, ...
Introduction to Research ,Need for research, Need for design of Experiments, ...Introduction to Research ,Need for research, Need for design of Experiments, ...
Introduction to Research ,Need for research, Need for design of Experiments, ...
 
physiotherapy in Acne condition.....pptx
physiotherapy in Acne condition.....pptxphysiotherapy in Acne condition.....pptx
physiotherapy in Acne condition.....pptx
 
Congestive Cardiac Failure..presentation
Congestive Cardiac Failure..presentationCongestive Cardiac Failure..presentation
Congestive Cardiac Failure..presentation
 
The role of Geography in climate education: science and active citizenship
The role of Geography in climate education: science and active citizenshipThe role of Geography in climate education: science and active citizenship
The role of Geography in climate education: science and active citizenship
 
How to Uninstall a Module in Odoo 17 Using Command Line
How to Uninstall a Module in Odoo 17 Using Command LineHow to Uninstall a Module in Odoo 17 Using Command Line
How to Uninstall a Module in Odoo 17 Using Command Line
 
Sulphonamides, mechanisms and their uses
Sulphonamides, mechanisms and their usesSulphonamides, mechanisms and their uses
Sulphonamides, mechanisms and their uses
 
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
 
Geoffrey Chaucer Works II UGC NET JRF TGT PGT MA PHD Entrance Exam II History...
Geoffrey Chaucer Works II UGC NET JRF TGT PGT MA PHD Entrance Exam II History...Geoffrey Chaucer Works II UGC NET JRF TGT PGT MA PHD Entrance Exam II History...
Geoffrey Chaucer Works II UGC NET JRF TGT PGT MA PHD Entrance Exam II History...
 
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...
 
Objectives n learning outcoms - MD 20240404.pptx
Objectives n learning outcoms - MD 20240404.pptxObjectives n learning outcoms - MD 20240404.pptx
Objectives n learning outcoms - MD 20240404.pptx
 
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 - I-LEARN SMART WORLD - CẢ NĂM - CÓ FILE NGHE (BẢN...
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 - I-LEARN SMART WORLD - CẢ NĂM - CÓ FILE NGHE (BẢN...BÀI TẬP BỔ TRỢ TIẾNG ANH 8 - I-LEARN SMART WORLD - CẢ NĂM - CÓ FILE NGHE (BẢN...
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 - I-LEARN SMART WORLD - CẢ NĂM - CÓ FILE NGHE (BẢN...
 
BÀI TẬP BỔ TRỢ 4 KĨ NĂNG TIẾNG ANH LỚP 8 - CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC ...
BÀI TẬP BỔ TRỢ 4 KĨ NĂNG TIẾNG ANH LỚP 8 - CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC ...BÀI TẬP BỔ TRỢ 4 KĨ NĂNG TIẾNG ANH LỚP 8 - CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC ...
BÀI TẬP BỔ TRỢ 4 KĨ NĂNG TIẾNG ANH LỚP 8 - CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC ...
 
Chi-Square Test Non Parametric Test Categorical Variable
Chi-Square Test Non Parametric Test Categorical VariableChi-Square Test Non Parametric Test Categorical Variable
Chi-Square Test Non Parametric Test Categorical Variable
 
PART 1 - CHAPTER 1 - CELL THE FUNDAMENTAL UNIT OF LIFE
PART 1 - CHAPTER 1 - CELL THE FUNDAMENTAL UNIT OF LIFEPART 1 - CHAPTER 1 - CELL THE FUNDAMENTAL UNIT OF LIFE
PART 1 - CHAPTER 1 - CELL THE FUNDAMENTAL UNIT OF LIFE
 
Grade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptxGrade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptx
 
Narcotic and Non Narcotic Analgesic..pdf
Narcotic and Non Narcotic Analgesic..pdfNarcotic and Non Narcotic Analgesic..pdf
Narcotic and Non Narcotic Analgesic..pdf
 
DBMSArchitecture_QueryProcessingandOptimization.pdf
DBMSArchitecture_QueryProcessingandOptimization.pdfDBMSArchitecture_QueryProcessingandOptimization.pdf
DBMSArchitecture_QueryProcessingandOptimization.pdf
 
Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...
Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...
Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...
 

Linear Algebra Seminar on Factorization and Transformations

  • 1. Seminar Series on Linear Algebra for Machine Learning Part 3: Factorization and Linear Transformations Dr. Ceni Babaoglu Data Science Laboratory Ryerson University cenibabaoglu.com Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 2. Overview 1 Row and Column Spaces 2 Rank of a Matrix 3 Rank and Singularity 4 Inner Product Spaces 5 Gram-Schmidt Process 6 Factorization 7 Linear Transformation 8 Linear Transformation and Singularity 9 Similar Matrices 10 References Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 3. Row and Column Spaces Let A =      a11 a12 a13 . . . a1n a21 a22 a23 . . . a2n ... ... ... ... ... am1 am2 am3 . . . amn      be an m × n matrix. The rows of A, considered as vectors in Rn, span a subspace of Rn called the row space of A. Similarly, the columns of A, considered as vectors in Rm, span a subspace of Rm called the column space of A. If A and B are two m × n row (column) equivalent matrices, then the row (column) spaces of A and B are equal. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 4. Rank of a Matrix The dimension of the row (column) space of A is called the row (column) rank of A. The row rank and column rank of the m × n matrix A = [aij ] are equal. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 5. Rank and Singularity A is nonsingular. Ax = 0 has only the trivial solution. A is row (column) equivalent to In. For every vector b in Rn, the system Ax = b has a unique solution. det(A) = 0. The rank of A is n. The rows of A form a linearly independent set of vectors in Rn. The columns of A form a linearly independent set of vectors in Rn. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 6. Example Let A =   1 −1 2 0 −3 0 1 0 4 0 2 −1 4 4 −6  . Find the following: (i) A basis for the column space of A and its dimension. (ii) A basis for the row space of A and its dimension. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 7. Example   1 −1 2 0 −3 0 1 0 4 0 2 −1 4 4 −6   S3−2S1→S3 −−−−−−−→   1 −1 2 0 −3 0 1 0 4 0 0 1 0 4 0   S1+S2→S1 S3−S2→S3 −−−−−−→   1 0 2 4 −3 0 1 0 4 0 0 0 0 0 0   (i) The column space of A is spanned by the vectors (1, 0, 2)T and (−1, 1, −1)T . These vectors are linearly independent. {(1, 0, 2)T , (−1, 1, −1)T } is a basis for this space and its dimension is 2. (ii) The row space of A is spanned by the vectors (1, 0, 2, 4, −3) and (0, 1, 0, 4, 0). These vectors are linearly independent. {(1, 0, 2, 4, −3), (0, 1, 0, 4, 0)} is a basis for this space and its dimension is 2. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 8. Inner Product Spaces Let V be a real vector space. An inner product on V is a function that assigns to each ordered pair of vectors u, v in V real number (u, v) satisfying the following properties: (u, u) ≥ 0, (u, u) = 0 if and only if u = 0v (v, u) = (u, v) for any u, v in V (u + v, w) = (u, w) + (v, w) for any u, v, w in V (cu, v) = c(u, v) for u, v in V and c, a real scalar A real vector space that has an inner product defined on it is called an inner product space. If the space is finite dimensional it is called a Euclidean space. In an inner product space we define the length of a vector u by u = (u, u). Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 9. Gram-Schmidt Process Let V be an inner product space and W = {0} an m-dimensional subspace of V . Then there exists an orthonormal basis T = {w1, w2, · · · , wm} for W . Let S = {u1, u2, · · · , um} be any basis for W . Construct an orthogonal basis T∗ = {v1, v2, · · · , vm} for W. Select any one of the vectors in S, say u1 and call it v1. Look for a vector v2 in the subspace W1 of W spanned by {u1, u2} that is orthogonal to v1. v2 = u2 − (u2, v1) (v1, v1) v1 Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 10. Gram-Schmidt Process Next, we look for a vector v3 in the subspace W2 of W spanned by {u1, u2, u3} that is orthogonal to both v1 and v2. v2 = u2 − (u2, v1) (v1, v1) v1 v3 = u3 − (u3, v1) (v1, v1) v1 − (u3, v2) (v2, v2) v2 Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 11. Example Let’s use the Gram-Schmidt process to find an orthonormal basis for the subspace of R4 with basis u1 = (1, 1, 1, 0)T , u2 = (−1, 0, −1, 1)T and u3 = (−1, 0, 0, −1)T . First let v1 = u1, v2 = u2 − (u2, v1) (v1, v1) v1 =     −1 0 −1 1     − (− 2 3 )     1 1 1 1     =     −1/3 2/3 −1/3 1     Multiplying v2 by 3 to clear fractions, we get     −1 2 −1 3     Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 12. Example v3 = u3 − (u3, v1) (v1, v1) v1 − (u3, v2) (v2, v2) v2 =     −4/5 3/5 1/5 −3/5     Multiplying v3 by 5 to clear fractions, we get (−4, 3, 1, −3)T . An orthogonal basis: {v1, v2, v3} =     1 1 1 0     ,     −1 2 −1 3     ,     −4 3 1 −3     An orthonormal basis: {w1, w2, w3} =     1/ √ 3 1/ √ 3 1/ √ 3 0     ,     −1/ √ 15 2/ √ 15 −1/ √ 15 3/ √ 15     ,     −4/ √ 35 3/ √ 35 1/ √ 35 −3/ √ 35     Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 13. Factorization If A is an m × n matrix with linearly independent columns, then A can be factored as A = QR, Q: an m × n matrix whose columns form an orthonormal basis for the column space of A, R: an n × n nonsingular upper triangular matrix. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 14. Example Let’s find the factorization of     1 −1 −1 1 0 0 1 −1 0 0 1 −1     . Let’s define the columns of A as the vectors u1, u2, u3. The orthonormal basis for the column space of A is w1 =     1/ √ 3 1/ √ 3 1/ √ 3 0     , w2 =     −1/ √ 15 2/ √ 15 −1/ √ 15 3/ √ 15     , w3 =     −4/ √ 35 3/ √ 35 1/ √ 35 −3/ √ 35     . Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 15. Example Q =     1/ √ 3 −1/ √ 15 −4/ √ 35 1/ √ 3 2/ √ 15 3/ √ 35 1/ √ 3 −1/ √ 15 1/ √ 35 0 3/ √ 15 −3/ √ 35     R =   r11 r12 r13 0 r22 r23 0 0 r33   where rji = (ui , wj ). R =   3/ √ 3 −2/ √ 3 −1/ √ 3 0 5/ √ 15 −2/ √ 15 0 0 7/ √ 35   A = QR Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 16. Linear Transformation A mapping L : V −→ W is said to be a linear transformation or a linear operator if L(αv1 + βv2) = αL(v1) + βL(v2) OR L(v1 + v2) = L(v1) + L(v2), (α = β = 1) L(αv) = αL(v) (v = v1, β = 0) Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 17. Example L(x) = 3x, x ∈ R2 . L(x + y) = 3(x + y) = L(x + y) L(αx) = 3(αx) = αL(x) L is a linear transformation. α : positive scalar F(x) = αx can be thought of as a stretching or shrinking by a factor of α. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 18. Example L(x) = x1e1, x ∈ R2 . If x = (x1, x2)T , then L(x) = (x, 0)T If y = (y1, y2)T , then αx + βy = αx1 + βy1 αx2 + βy2 L(αx + βy) = (αx1 + βy1)e1 = α(x1e1) + β(y1e1) = αL(x) + βL(y) L is a linear transformation, a projection onto the x1 axis. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 19. Example L(x) = (−x2, x1)T , x = (x1, x2)T ∈ R2. L(αx + βy) = −(αx2 + βy2) αx1 + βy1 = α −x2 x1 + β −y2 y1 = αL(x) + βL(y) L is a linear transformation. It has the effect of rotating each vector in R2 by 90◦ in the counterclockwise direction. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 20. Example M(x) = (x2 1 + x2 2 )1/2, M : R2 −→ R. M(αx) = (α2 x2 1 + α2 x2 2 )1/2 =| α | M(x), αM(x) = M(αx), α < 0, x = 0. M is not a linear transformation. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 21. Linear One-to-one Transformations A linear transformation L : V → W is called one-to-one if it is a one-to-one function; that is, if v1 = v2 implies that L(v1) = L(v2). An equivalent statement is that L is one-to-one if L(v1) = L(v2) implies that v1 = v2. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 22. Linear Onto Transformations If L : V → W is a linear transformation of a vector space V into a vector space W , then the range of L or image of V under L, denoted by range L , consists of all those vectors in W that are images under L of vectors in V . Thus w is in range L if there exists some vector v in V such that L(v) = w. The linear transformation L is called onto if range L = W . Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 23. Linear Transformation and Singularity A is nonsingular. Ax = 0 has only the trivial solution. A is row (column) equivalent to In. For every vector b in Rn, the system Ax = b has a unique solution. det(A) = 0. The rank of A is n. The rows of A form a linearly independent set of vectors in Rn. The columns of A form a linearly independent set of vectors in Rn. The linear transformation L : Rn −→ Rn defined by L(x) = A(x), for x in Rn, is one-to-one and onto. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 24. Similar Matrices If A and B are n × n matrices, we say that B is similar to A if there is a nonsingular matrix P such that B = P−1AP. Let V be any n−dimensional vector space and let A and B be any n × n matrices. Then A and B are similar if and only if A and B represent the same linear transformation L : V → V with respect to two ordered bases for V . If A and B are similar n × n matrices, then rank A = rank B. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 25. Example Let L : R3 → R3 be defined by L([ u1 u2 u3 ]) = [ 2u1 − u3 u1 + u2 − u3 u3 ] and S = {[1 0 0], [0 1 0], [0 0 1]} be the natural basis for R3. The representation of L with respect of S is A =   2 0 −1 1 1 −1 0 0 1   . Considering S = {[1 0 1], [0 1 0], [1 1 0]} as ordered bases for R3, the transition matrix P from S to S is P =   1 0 1 0 1 1 1 0 0   P−1 =   0 0 1 −1 1 1 1 0 −1   . Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 26. Example Then the representation of L with respect to S is B = P−1 AP =   1 0 0 0 1 0 0 0 2   The matrices A =   2 0 −1 1 1 −1 0 0 1   and B =   1 0 0 0 1 0 0 0 2   are similar. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations
  • 27. References Linear Algebra With Applications, 7th Edition by Steven J. Leon. Elementary Linear Algebra with Applications, 9th Edition by Bernard Kolman and David Hill. Dr. Ceni Babaoglu cenibabaoglu.com Linear Algebra for Machine Learning: Factorization and Linear Transformations