Linear algebra notes

Linear Algebra & NumericalLinear Algebra & Numerical
AnalysisAnalysis
Nek MUHAMMADNek MUHAMMAD
ASSISTANT PROFESSORASSISTANT PROFESSOR
Department of BSRSDepartment of BSRS
MEHRAN UET KHAIRPUR MIR’SMEHRAN UET KHAIRPUR MIR’S
0331396899603313968996

INTRODUCTIONINTRODUCTION
The termThe term linear algebralinear algebra was first used in the modernwas first used in the modern
sense bysense by van der Waerden (1870)van der Waerden (1870)
Linear algebra is the study of linear sets of equations and theirLinear algebra is the study of linear sets of equations and their
transformation properties. Linear algebra allows the analysistransformation properties. Linear algebra allows the analysis
ofof rotations in space, least squares fittings solution of coupledrotations in space, least squares fittings solution of coupled
differential equations, determination of a circle passing through threedifferential equations, determination of a circle passing through three
given points, as well as many other problems in mathematics,given points, as well as many other problems in mathematics,
physics, and engineering.physics, and engineering.
OROR
 a branch of mathematics that is concerned with mathematicala branch of mathematics that is concerned with mathematical
structures closed under the operations of addition and scalarstructures closed under the operations of addition and scalar
multiplication and that includes the theory of systems of linearmultiplication and that includes the theory of systems of linear
equations, matrices, determinants, vector spaces, and linearequations, matrices, determinants, vector spaces, and linear
transformationstransformations

APPLICATIONS OF LINREAR ALGEBRAAPPLICATIONS OF LINREAR ALGEBRA
• Linear Algebra is used in various fields of Science andLinear Algebra is used in various fields of Science and
Technology.Technology.
• Chemistry – Coding Theory – Cryptography – Economics –Chemistry – Coding Theory – Cryptography – Economics –
Elimination Theory – Games – Genetics – Geometry – GraphElimination Theory – Games – Genetics – Geometry – Graph
Theory – Heat Distribution – Image Compression – LinearTheory – Heat Distribution – Image Compression – Linear
Programming – Markov Chains – Networking – Sociology – TheProgramming – Markov Chains – Networking – Sociology – The
Fibonacci Numbers – Eigenfaces and many more…Fibonacci Numbers – Eigenfaces and many more…..
1.Chemical Applications › Application of linear systems to chemistry is1.Chemical Applications › Application of linear systems to chemistry is
balancing a chemical equation and also finding the volume ofbalancing a chemical equation and also finding the volume of
substance. The rationale behind this is the Law of conservation ofsubstance. The rationale behind this is the Law of conservation of
mass which states the following: › “Mass is neither created normass which states the following: › “Mass is neither created nor
destroyed in any chemical reaction. Therefore balancing of equationsdestroyed in any chemical reaction. Therefore balancing of equations
requires the same number of atoms on both sides of a chemicalrequires the same number of atoms on both sides of a chemical
reaction. The mass of all the reactants (the substances going into areaction. The mass of all the reactants (the substances going into a
reaction) must equal the mass of the products (the substancesreaction) must equal the mass of the products (the substances
produced by the reaction.produced by the reaction.

• As an example consider the following chemicalAs an example consider the following chemical
equation C2H6 + O2 CO2 + H2O. Balancing this→equation C2H6 + O2 CO2 + H2O. Balancing this→
chemical reaction means finding values of x, y, z andchemical reaction means finding values of x, y, z and
t so that the number of atoms of each element is thet so that the number of atoms of each element is the
same on both sides of the equation: xC2H6 + yO2 →same on both sides of the equation: xC2H6 + yO2 →
zCO2 + tH2O. This gives the following linear system:zCO2 + tH2O. This gives the following linear system:
The general solution of the above system is: SinceThe general solution of the above system is: Since
we are looking for whole values of the variables x, ywe are looking for whole values of the variables x, y
z, and t, choose x=2 and get y=7, z= 4 and t=6.z, and t, choose x=2 and get y=7, z= 4 and t=6.
The balanced equation is then: 2C2H 6 + 7O2 →The balanced equation is then: 2C2H 6 + 7O2 →
4CO2 + 6H2O4CO2 + 6H2O..
2.2. Applications in Coding Theory TransmittedApplications in Coding Theory Transmitted
messages, like data from a satellite, are alwaysmessages, like data from a satellite, are always
subject to noise. It is important; therefore, to besubject to noise. It is important; therefore, to be
able to encode a message in such a way that afterable to encode a message in such a way that after
noise scrambles itnoise scrambles it

it can be decoded to its original form. This is done sometimes byit can be decoded to its original form. This is done sometimes by
repeating the message two or three times, something very common inrepeating the message two or three times, something very common in
human speech. However, copying data stored on a compact disk, or ahuman speech. However, copying data stored on a compact disk, or a
floppy disk once or twice requires extra space to store.floppy disk once or twice requires extra space to store.
In this application, we will examine ways of decoding a message after itIn this application, we will examine ways of decoding a message after it
gets distorted by some kind of noise. This process is called coding. Agets distorted by some kind of noise. This process is called coding. A
code that detects errors in a scrambled message is called errorcode that detects errors in a scrambled message is called error
detecting. If, in addition, it can correct the error it is called errordetecting. If, in addition, it can correct the error it is called error
correcting. It is much harder to find error correcting than error-correcting. It is much harder to find error correcting than error-
detecting codes.detecting codes.
3.3. Coupled Oscillations › Everyone unconsciously knows this Law.Coupled Oscillations › Everyone unconsciously knows this Law.
Everyone knows that heavier objects require more force to move theEveryone knows that heavier objects require more force to move the
same distance than do lighter objects. The Second Law, however, givessame distance than do lighter objects. The Second Law, however, gives
us an exact relationship between force, mass, and acceleration: ›us an exact relationship between force, mass, and acceleration: ›

In the presence of external forces, an object experiences anIn the presence of external forces, an object experiences an
acceleration directly proportional to the net external force andacceleration directly proportional to the net external force and
inversely proportional to the mass of the object. › This Law Isinversely proportional to the mass of the object. › This Law Is
widely known with the following equation: F = ma.widely known with the following equation: F = ma.
This law when used with Hooke’s Second Law helps to find theThis law when used with Hooke’s Second Law helps to find the
oscillations of coupled springs arranged in various examplesoscillations of coupled springs arranged in various examples
4.4. Cryptography, to most people, is concerned with keepingCryptography, to most people, is concerned with keeping
communications private. Indeed, the protection of sensitivecommunications private. Indeed, the protection of sensitive
communications has been the emphasis of cryptographycommunications has been the emphasis of cryptography
throughout much of its history. › Encryption is the transformationthroughout much of its history. › Encryption is the transformation
of data into some unreadable form. Its purpose is to ensure privacyof data into some unreadable form. Its purpose is to ensure privacy
by keeping the information hidden from anyone for whom it is notby keeping the information hidden from anyone for whom it is not
intended, even those who can see the encrypted dataintended, even those who can see the encrypted data

5.5. Encryption and decryption require the use of some secretEncryption and decryption require the use of some secret
information, usually referred to as a key. Depending on theinformation, usually referred to as a key. Depending on the
encryption mechanism used, the same key might be used for bothencryption mechanism used, the same key might be used for both
encryption and decryption, while for other mechanisms, the keysencryption and decryption, while for other mechanisms, the keys
used for encryption and decryption might be different.used for encryption and decryption might be different.
Today governments use sophisticated methods of coding andToday governments use sophisticated methods of coding and
decoding messages. One type of code, which is extremely difficultdecoding messages. One type of code, which is extremely difficult
to break, makes use of a large matrix to encode a message. Theto break, makes use of a large matrix to encode a message. The
receiver of the message decodes it using the inverse of the matrix.receiver of the message decodes it using the inverse of the matrix.
This first matrix is called the encoding matrix and its inverse isThis first matrix is called the encoding matrix and its inverse is
called the decoding matrix. It is used in ATM cards ,MOBILEcalled the decoding matrix. It is used in ATM cards ,MOBILE
passwords, COMPUTER locks, SUPER CARDS and MOBILEpasswords, COMPUTER locks, SUPER CARDS and MOBILE
cards etc.cards etc.

6.Applications in various GAMES › GAME OF MAGIC6.Applications in various GAMES › GAME OF MAGIC
SQUARES: › A magic square of size n is an n by nSQUARES: › A magic square of size n is an n by n
square matrix whose entries consist of all integerssquare matrix whose entries consist of all integers
between 1 and n2, with the property that the sum ofbetween 1 and n2, with the property that the sum of
the entries of each column, row, or diagonal is thethe entries of each column, row, or diagonal is the
same. › The sum of the entries of any row, column,same. › The sum of the entries of any row, column,
or diagonal, of a magic square of size n is n(n2+1)/2or diagonal, of a magic square of size n is n(n2+1)/2
(to see this, use the identity: 1+2+...+k=k(k+1)/2).(to see this, use the identity: 1+2+...+k=k(k+1)/2).
7.Application to Genetics › Living things inherit from7.Application to Genetics › Living things inherit from
their parents many of their physical characteristics.their parents many of their physical characteristics.
The genes of the parents determine theseThe genes of the parents determine these
characteristics. The study of these genes is calledcharacteristics. The study of these genes is called
Genetics; in other words genetics is the branch ofGenetics; in other words genetics is the branch of
biology that deals with heredity.biology that deals with heredity.

• ››In particular, population genetics is the branch of genetics thatIn particular, population genetics is the branch of genetics that
studies the genetic structure of a certain population and seeks tostudies the genetic structure of a certain population and seeks to
explain how transmission of genes changes from one generation toexplain how transmission of genes changes from one generation to
another. Genes govern the inheritance of traits like sex, color of theanother. Genes govern the inheritance of traits like sex, color of the
eyes, hair (for humans and animals), leaf shape and petal color (foreyes, hair (for humans and animals), leaf shape and petal color (for
plants). › There are several types of inheritance; one of particularplants). › There are several types of inheritance; one of particular
interest for us is the autosomal type in which each heritable trait isinterest for us is the autosomal type in which each heritable trait is
assumed to be governed by a single geneassumed to be governed by a single gene.. Typically, there are twoTypically, there are two
different forms of genes denoted by A and a. › Each individual in adifferent forms of genes denoted by A and a. › Each individual in a
population carries a pair of genes; the pairs are called the individual’spopulation carries a pair of genes; the pairs are called the individual’s
genotype. This gives three possible genotypes for each inheritablegenotype. This gives three possible genotypes for each inheritable
trait: AA, Aa, and aa .trait: AA, Aa, and aa . in a certain animal population, an autosomalin a certain animal population, an autosomal
model of inheritance controls eye coloration. Genotypes AA and Aamodel of inheritance controls eye coloration. Genotypes AA and Aa
have brown eyes, while genotype aa has blue eyes. The A gene is saidhave brown eyes, while genotype aa has blue eyes. The A gene is said
to dominate the a gene. An animal is called dominant if it has AAto dominate the a gene. An animal is called dominant if it has AA
genes, hybrid with Aa genes, and recessive with aa genes.genes, hybrid with Aa genes, and recessive with aa genes.

• This means that genotypes AA and Aa areThis means that genotypes AA and Aa are
indistinguishable in appearance. › Each offspringindistinguishable in appearance. › Each offspring
inherits one gene from each parent in a randominherits one gene from each parent in a random
manner. Given the genotypes of the parents, we canmanner. Given the genotypes of the parents, we can
determine the probabilities of the genotype of thedetermine the probabilities of the genotype of the
offspring. Suppose that, in this animal population,offspring. Suppose that, in this animal population,
the initial distribution of genotypes is given by thethe initial distribution of genotypes is given by the
vector is called the transition matrix.vector is called the transition matrix.
In general, Xn =AXn-1In general, Xn =AXn-1..
9.9. GEOMETRICAL APPLICATIONS › Given some fixedGEOMETRICAL APPLICATIONS › Given some fixed
points in the plane or in 3-D space, many problemspoints in the plane or in 3-D space, many problems
require finding some geometric figures passingrequire finding some geometric figures passing
through these points. Distances, Eq of st: linesthrough these points. Distances, Eq of st: lines
,direction ratios, direction QIBLA ,Eq of panes etc,direction ratios, direction QIBLA ,Eq of panes etc

 Linear Algebra in software engineeringLinear Algebra in software engineering
 linear algebra is crucial to:linear algebra is crucial to:
Audio, video and image compression, including MP3, JPEG/JPEG-Audio, video and image compression, including MP3, JPEG/JPEG-
2000 and MPEG video or VP8 and computer graphics2000 and MPEG video or VP8 and computer graphics
 Modulation and coding, including convolutional codes e.g., EV-DO,Modulation and coding, including convolutional codes e.g., EV-DO,
Wi-Fi, Gigabit Ethernet, QAM, HDTV and the Global PositioningWi-Fi, Gigabit Ethernet, QAM, HDTV and the Global Positioning
SystemSystem
 Signal processing, including the Fast Fourier Transform and autotune!Signal processing, including the Fast Fourier Transform and autotune!
"I'm on a Boat" would not have been possible without linear algebra"I'm on a Boat" would not have been possible without linear algebra

• One of the most important applications of linear algebra toOne of the most important applications of linear algebra to
electronics is to analyze electronic circuits that cannot beelectronics is to analyze electronic circuits that cannot be
described using the rules for resistors in series or parallel such asdescribed using the rules for resistors in series or parallel such as
the one shown to the right. The goal is to calculate the currentthe one shown to the right. The goal is to calculate the current
flowing in each branch of the circuit or to calculate the voltage atflowing in each branch of the circuit or to calculate the voltage at
each node of the circuit.each node of the circuit.
• Linear algebra is used in engineering to solve problems thatLinear algebra is used in engineering to solve problems that
involve multiple complex equations. In the petroleum industry,involve multiple complex equations. In the petroleum industry,
engineers must model reservoirs to find the conditions needed toengineers must model reservoirs to find the conditions needed to
maximize recovery of hydrocarbons. Reservoir simulationmaximize recovery of hydrocarbons. Reservoir simulation
involves many complex partial differential equations, which areinvolves many complex partial differential equations, which are
solved using a computer program that models the reservoir beingsolved using a computer program that models the reservoir being
studiedstudied

To the petroleum engineer it is great that the computer program doesTo the petroleum engineer it is great that the computer program does
all the simulating for him as long as all the correct parameters areall the simulating for him as long as all the correct parameters are
put into the program. We will look at how reservoir simulators useput into the program. We will look at how reservoir simulators use
linear algebra to model reservoirs.linear algebra to model reservoirs.
The main tool of linear algebra isThe main tool of linear algebra is MATRICESMATRICES && DETERMINANTSDETERMINANTS

MatricesMatrices
 The idea of Matrix is given by Arthur Kelly in 1858.The idea of Matrix is given by Arthur Kelly in 1858.
The matrix is used by J.J Sylvester in linearThe matrix is used by J.J Sylvester in linear
transformation.transformation.
 Matrix is Latin word used for WOMB or a place inMatrix is Latin word used for WOMB or a place in
which some thing develops or formswhich some thing develops or forms
 A matrix is a rectangular array of numbers (orA matrix is a rectangular array of numbers (or
functions). Or Arrangements of numbers in rowsfunctions). Or Arrangements of numbers in rows
and columns .and columns .
 MatrixMatrix is a collection of numbers arranged into ais a collection of numbers arranged into a
fixed number of rows and columnsfixed number of rows and columns
 Matrix is set of elements in rows and columns.Matrix is set of elements in rows and columns.
Matrix is common device of summarizing data.Matrix is common device of summarizing data.

MatricesMatrices
The size or dimension of a matrix is defined by theThe size or dimension of a matrix is defined by the
number of rows and columns it contains.number of rows and columns it contains.
The matrix shown above is of size mThe matrix shown above is of size mxxn. Note that thisn. Note that this
designates first the number of rows, then thedesignates first the number of rows, then the
number of columns.number of columns.
 The elements of a matrix, here represented by theThe elements of a matrix, here represented by the
letter ‘a’ with subscripts, can consist of numbers,letter ‘a’ with subscripts, can consist of numbers,
variables, or functions of variables.variables, or functions of variables.












mnm
n
aa
aa
aaa
1
2221
11211



VectorsVectors
 A vector is simply a matrix with either one row orA vector is simply a matrix with either one row or
one column.one column. A matrix with one row is called a rowA matrix with one row is called a row
vector, and a matrix with one column is called avector, and a matrix with one column is called a
column vectorcolumn vector..
 Transpose: A row vector can be changed into aTranspose: A row vector can be changed into a
column vector and vice-versa by taking thecolumn vector and vice-versa by taking the
transposetranspose of that vector. e.g.:of that vector. e.g.:
[ ]










==
5
4
3
543 T
AthenAif

TYPES OF MATRIXTYPES OF MATRIX
 A matrix may be classified by types. It is possible for a matrix to belong toA matrix may be classified by types. It is possible for a matrix to belong to
more than one type.more than one type.
 AA row matrixrow matrix is a matrix with only one row.is a matrix with only one row.
 AA column matrixcolumn matrix is a matrix with only one column.is a matrix with only one column.
 AA zero matrixzero matrix or aor a null matrixnull matrix is a matrix that has all its elements zero.is a matrix that has all its elements zero.
 AA square matrixsquare matrix is a matrix with an equal number of rows and columns.is a matrix with an equal number of rows and columns.
 AA diagonal matrixdiagonal matrix is a square matrix that has all its elements zero exceptis a square matrix that has all its elements zero except
for those in the diagonal from top left to bottom right; which is known asfor those in the diagonal from top left to bottom right; which is known as
thethe leading diagonalleading diagonal of the matrixof the matrix..
 AA scalar matrixscalar matrix is a diagonal matrix where all the diagonal elements areis a diagonal matrix where all the diagonal elements are
equalequal
 AnAn upper triangular matrixupper triangular matrix is a square matrix where all the elementsis a square matrix where all the elements
located below the diagonal are zeroslocated below the diagonal are zeros

 AA lower triangular matrixlower triangular matrix is a square matrix where all the elementsis a square matrix where all the elements
located above the diagonal are zeros.located above the diagonal are zeros.
 A matrix which is upper as well as lower is called triangular matrixA matrix which is upper as well as lower is called triangular matrix
 AA unit matrix or identity matrixunit matrix or identity matrix is a diagonal matrix whoseis a diagonal matrix whose
elements in the diagonal are all ones.elements in the diagonal are all ones.
 Transpose of a matrixTranspose of a matrix. The matrix resulting from interchanging the rows. The matrix resulting from interchanging the rows
and columns of matrix.and columns of matrix.
 Symmetric matrix.Symmetric matrix. A square matrix in which correspondingA square matrix in which corresponding
elements with respect to the diagonal are equal; a matrix inelements with respect to the diagonal are equal; a matrix in
which awhich aijij = a= ajiji where awhere aijij is the element in the i-th row and j-this the element in the i-th row and j-th
column; a matrix which is equal to its transpose; a square matrixcolumn; a matrix which is equal to its transpose; a square matrix
in which a flip about the diagonal leaves it unchangedin which a flip about the diagonal leaves it unchanged
t
A A=

 Skew-symmetric matrixSkew-symmetric matrix. A square matrix in which. A square matrix in which
corresponding elements with respect to the diagonal arecorresponding elements with respect to the diagonal are
negatives of each other; a matrix in which anegatives of each other; a matrix in which aijij = -a= -ajiji where awhere aijij isis
the element in the i-th row and j-th column; a matrix which isthe element in the i-th row and j-th column; a matrix which is
equal to the negative of its transpose. The diagonal elements areequal to the negative of its transpose. The diagonal elements are
always zerosalways zeros
 Periodic matrix.Periodic matrix. A matrix A for whichA matrix A for which AAk+1k+1
= A= A , where k is a, where k is a
positive integer. If k is the least positive integer for which Apositive integer. If k is the least positive integer for which Ak+1k+1
==
A , then A is said to be ofA , then A is said to be of period k.period k.
 Idempotent matrixIdempotent matrix if Aif A22
= A, then A is called= A, then A is called idempotentidempotent..
 Nilpotent matrixNilpotent matrix. A matrix A for which. A matrix A for which AApp
= 0= 0, where p is some, where p is some
positive integer. If p is the least positive integer for which Apositive integer. If p is the least positive integer for which App
= 0, then A= 0, then A
is said to beis said to be nilpotent of index pnilpotent of index p..

t
A A= −

Matrix AdditionMatrix Addition
 Matrix addition is only possible betweenMatrix addition is only possible between
two matrices which have the same size.two matrices which have the same size.
 The operation is done simply by addingThe operation is done simply by adding
the corresponding elements. e.g.:the corresponding elements. e.g.:






=





+





87
57
13
26
74
31

Matrix scalar multiplicationMatrix scalar multiplication
 Multiplication of a matrix or a vectorMultiplication of a matrix or a vector
by a scalar is also straightforwardby a scalar is also straightforward::






=





3520
155
74
31
*5

Transpose of a matrixTranspose of a matrix










=










=
028
573
641
,
056
274
831
T
AthenAif
 Taking the transpose of a matrix is similarTaking the transpose of a matrix is similar
to that of a vector:to that of a vector:
 The diagonal elements in the matrix areThe diagonal elements in the matrix are
unaffected, but the other elements areunaffected, but the other elements are
switchedswitched. A matrix which is the same as. A matrix which is the same as
its own transpose is calledits own transpose is called symmetricsymmetric, and, and
one which is the negative of its ownone which is the negative of its own
transpose is calledtranspose is called skew-symmetricskew-symmetric..

Matrix MultiplicationMatrix Multiplication
 The multiplication of a matrix into another matrixThe multiplication of a matrix into another matrix
not possible for all matrices, and the operation isnot possible for all matrices, and the operation is
not commutativenot commutative::
AB ≠ BA in generalAB ≠ BA in general
 In order to multiply two matrices, the first matrixIn order to multiply two matrices, the first matrix
must have the same number of columns as themust have the same number of columns as the
second matrix has rows.second matrix has rows.
 So, if one wants to solve for C=AB, then theSo, if one wants to solve for C=AB, then the
matrix A must have as many columns as thematrix A must have as many columns as the
matrix B has rows.matrix B has rows.
 The resulting matrix C will have the same numberThe resulting matrix C will have the same number
of rows as did A and the same number ofof rows as did A and the same number of
columns as did B.columns as did B.

Matrix MultiplicationMatrix Multiplication
 The operation is done as follows:The operation is done as follows:
using index notation:using index notation:
for example:for example:
lk
n
l jljk BAC ∑=
= 1










=










++
++
++
=















=
4518
4716
3811
6*05*91*02*9
6*25*71*22*7
6*35*41*32*4
61
52
09
27
34
AB

ELEMENTARY ROW OPERATION (ERO)ELEMENTARY ROW OPERATION (ERO)
There are three types of EROThere are three types of ERO
1.1. Interchange two rows.Interchange two rows.
2.2.Multiply a row with a nonzeroMultiply a row with a nonzero
number.number.
3.3. Add a row to another one multipliedAdd a row to another one multiplied
by a number.by a number.
If these operations are performed toIf these operations are performed to
rows then it is called EROrows then it is called ERO

ECHELON FORM
 A matrix is in echelon form (or row echelon
form) if it has the following three properties:
1. All nonzero rows are above any rows of all
zeros.
2. Each leading entry of a row is in a column to
the right of the leading entry of the row above
it.
3. All entries in a column below a leading entry
are zeros.

REDUCED ECHELON FORM
 If a matrix in echelon form satisfies the following
additional conditions, then it is in reduced echelon
form (or reduced row echelon form):
4. The leading entry in each nonzero row is 1.
5. Each leading 1 is the only nonzero entry in its
column.
 An echelon matrix (respectively, reduced echelon
matrix) is one that is in echelon form (respectively,
reduced echelon form.)

ECHELON &REDUCED ECHELONFORM
 Any nonzero matrix may be row reduced (i.e.,
transformed by elementary row operations) into more
than one matrix in echelon form, using different
sequences of row operations. However, the reduced
echelon form one obtains from a matrix is unique.
Theorem 1: Uniqueness of the Reduced Echelon
Form
Each matrix is row equivalent to one and only one
reduced echelon matrix.

PIVOT POSITION
 If a matrix A is row equivalent to an echelon matrix
U, we call U an echelon form (or row echelon form)
of A; if U is in reduced echelon form, we call U the
reduced echelon form of A.
 A pivot position in a matrix A is a location in A that
corresponds to a leading 1 in the reduced echelon
form of A. A pivot column is a column of A that
contains a pivot position.

PIVOT POSITION
 Example 1: Row reduce the matrix A below to
echelon form, and locate the pivot columns of A.
 Solution: The top of the leftmost nonzero column is
the first pivot position. A nonzero entry, or pivot,
must be placed in this position.
0 3 6 4 9
1 2 1 3 1
2 3 0 3 1
1 4 5 9 7
A
− − 
 − − −
 =
− − − 
 − − 

PIVOT POSITION
 Now, interchange rows 1 and 4.
 Create zeros below the pivot, 1, by adding multiples
of the first row to the rows below, and obtain the next
matrix.
1 4 5 9 7
1 2 1 3 1
2 3 0 3 1
0 3 6 4 9
− − 
 − − −
 
− − − 
 − − 
Pivot
Pivot column

PIVOT POSITION
 Choose 2 in the second row as the next pivot.
 Add times row 2 to row 3, and add times
row 2 to row 4.
1 4 5 9 7
0 2 4 6 6
0 5 10 15 15
0 3 6 4 9
− − 
 − −
 
− − 
 − − 
Pivot
Next pivot column
5 / 2− 3/ 2

PIVOT POSITION
 There is no way a leading entry can be created in
column 3. But, if we interchange rows 3 and 4, we
can produce a leading entry in column 4.
1 4 5 9 7
0 2 4 6 6
0 0 0 0 0
0 0 0 5 0
− − 
 − −
 
 
 − 

PIVOT POSITION
1 4 5 9 7
0 2 4 6 6
0 0 0 5 0
0 0 0 0 0
− − 
 − −
 
− 
 
 
 The matrix is in echelon form and thus reveals that
columns 1, 2, and 4 of A are pivot columns.
Pivot
Pivot columns

PIVOT POSITION
0 3 6 4 9
1 2 1 3 1
2 3 0 3 1
1 4 5 9 7
A
− − 
 − − −
 =
− − − 
 − − 
 The pivots in the example are 1, 2 and .
Pivot positions
Pivot columns
5−

ROW REDUCTION ALGORITHM
 Example : Apply elementary row operations to
transform the following matrix first into echelon form
and then into reduced echelon form.
 Solution:
 STEP 1: Begin with the leftmost nonzero column.
This is a pivot column. The pivot position is at the
top.
0 3 6 6 4 5
3 7 8 5 8 9
3 9 12 9 6 15
− − 
 − −
 
− −  

 STEP 2: Select a nonzero entry in the pivot column as a
pivot. If necessary, interchange rows to move this entry
into the pivot position.
0 3 6 6 4 5
3 7 8 5 8 9
3 9 12 9 6 15
− − 
 − −
 
− −  
Pivot column

 Interchange rows 1 and 3. (Rows 1 and 2 could have
also been interchanged instead.)
 STEP 3: Use row replacement operations to create
zeros in all positions below the pivot.
3 9 12 9 6 15
3 7 8 5 8 9
0 3 6 6 4 5
− − 
 − −
 
− −  
Pivot

 We could have divided the top row by the pivot, 3, but
with two 3s in column 1, it is just as easy to add
times row 1 to row 2.
 STEP 4: Cover the row containing the pivot position,
and cover all rows, if any, above it. Apply steps 1–3 to
the sub matrix that remains. Repeat the process until
there are no more nonzero rows to modify.
1−
3 9 12 9 6 15
0 2 4 4 2 6
0 3 6 6 4 5
− − 
 − −
 
− −  
Pivot

 With row 1 covered, step 1 shows that column 2 is the
next pivot column; for step 2, select as a pivot the “top”
entry in that column.
 For step 3, we could insert an optional step of dividing
the “top” row of the submatrix by the pivot, 2. Instead,
we add times the “top” row to the row below.
3 9 12 9 6 15
0 2 4 4 2 6
0 3 6 6 4 5
− − 
 − −
 
− −  
Pivot
New pivot column
3/ 2−

 This produces the following matrix.
 When we cover the row containing the second pivot
position for step 4, we are left with a new submatrix that
has only one row.
3 9 12 9 6 15
0 2 4 4 2 6
0 0 0 0 1 4
− − 
 − −
 
  
3 9 12 9 6 15
0 2 4 4 2 6
0 0 0 0 1 4
− − 
 − −
 
  

 Steps 1–3 require no work for this sub matrix, and we
have reached an echelon form of the full matrix. We
perform one more step to obtain the reduced echelon
form.
 STEP 5: Beginning with the rightmost pivot and
working upward and to the left, create zeros above
each pivot. If a pivot is not 1, make it 1 by a scaling
operation.
 The rightmost pivot is in row 3. Create zeros above it,
adding suitable multiples of row 3 to rows 2 and 1.

3 9 12 9 0 9
0 2 4 4 0 14
0 0 0 0 1 4
− − − 
 − −
 
  
 The next pivot is in row 2. Scale this row, dividing by
the pivot.
Row 1 ( 6) row 3+ − ×
Row 2 ( 2) row 3+ − ×
3 9 12 9 0 9
0 1 2 2 0 7
0 0 0 0 1 4
− − − 
 − −
 
  
1
Row scaled by
2

 Create a zero in column 2 by adding 9 times row 2 to
row 1.
 Finally, scale row 1, dividing by the pivot, 3.
Row 1 (9) row 2+ ×3 0 6 9 0 72
0 1 2 2 0 7
0 0 0 0 1 4
− − 
 − −
 
  

 This is the reduced echelon form of the original
matrix.
 The combination of steps 1–4 is called the forward
phase of the row reduction algorithm. Step 5, which
produces the unique reduced echelon form, is called
the backward phase.
1 0 2 3 0 24
0 1 2 2 0 7
0 0 0 0 1 4
− − 
 − −
 
  
1
Row scaled by
3

MATRIX RANKMATRIX RANK
 The rank of a matrix is simply the number ofThe rank of a matrix is simply the number of
NON ZEROSNON ZEROS rows in matrix after convertingrows in matrix after converting
it intoit into Echelon FORMEchelon FORM..
 The transpose of a matrix has the same rankThe transpose of a matrix has the same rank
as the original matrix.as the original matrix.
 To find the rank of a matrix by hand, useTo find the rank of a matrix by hand, use
Gauss elimination methodGauss elimination method..

MATRIX INVERSEMATRIX INVERSE
 The inverse of the matrix A is denoted as AThe inverse of the matrix A is denoted as A-1-1
 By definition, AABy definition, AA-1-1
= A= A-1-1
A = I, where I is theA = I, where I is the
identity matrix.identity matrix.
 Theorem: The inverse of an nTheorem: The inverse of an n xx n matrix An matrix A
exists if and only if the rank A = n.exists if and only if the rank A = n.
 Gauss-Jordan elimination can be used to findGauss-Jordan elimination can be used to find
the inverse of a matrix by handthe inverse of a matrix by hand..

Special matricesSpecial matrices
 A matrix is calledA matrix is called symmetricsymmetric if:if:
AATT
= A= A
 A skew-symmetric matrix is one forA skew-symmetric matrix is one for
which:which:
AATT
= -A= -A
 AnAn orthogonalorthogonal matrix is one whosematrix is one whose
transpose is also its inverse:transpose is also its inverse:
AATT
= A= A-1-1

Complex matricesComplex matrices
 If a matrix contains complex (imaginary)If a matrix contains complex (imaginary)
elements, it is often useful to take itselements, it is often useful to take its
complex conjugatecomplex conjugate.. The notation used forThe notation used for
the complex conjugate of a matrix A is:the complex conjugate of a matrix A is:
A^-A^-
 Some special complex matrices are asSome special complex matrices are as
follows:follows:
Hermitian:Hermitian: A-A-TT
= A= A
Skew-Hermitian:Skew-Hermitian: A-A-TT
= -A= -A
Unitary:Unitary: A-A-TT
= A= A-1-1

DeterminantsDeterminants
 Determinants are useful in eigenvalueDeterminants are useful in eigenvalue
problems and differential equations.problems and differential equations.
 Can be found only for square matrices.Can be found only for square matrices.
 Simple example: 2Simple example: 2ndnd
order determinantorder determinant
54*37*1
74
31
det −=−==A

33rdrd
order determinantorder determinant
 The determinant of a 3The determinant of a 3XX3 matrix is3 matrix is
found as follows:found as follows:
 The terms on the RHS can beThe terms on the RHS can be
evaluated as shown for a 2evaluated as shown for a 2ndnd
orderorder
determinant.determinant.
3231
2221
13
3331
2321
12
3332
2322
11
333231
232221
131211
det
aa
aa
a
aa
aa
a
aa
aa
a
aaa
aaa
aaa
A +−==

Some theorems for determinantsSome theorems for determinants
 Cramer’s: If the determinant of aCramer’s: If the determinant of a
system of n equations with nsystem of n equations with n
unknowns is nonzero, that systemunknowns is nonzero, that system
has precisely one solution.has precisely one solution.
 det(AB)=det(BA)=det(A)det(B)det(AB)=det(BA)=det(A)det(B)

Eigenvalues and EigenvectorsEigenvalues and Eigenvectors
 Let A be an nxn matrix and consider theLet A be an nxn matrix and consider the
vector equation:vector equation:
Ax =Ax = λλxx
 A value ofA value of λλ for which this equation has afor which this equation has a
solution x≠0 is called an eigenvalue of thesolution x≠0 is called an eigenvalue of the
matrix A.matrix A.
 The corresponding solutions x are calledThe corresponding solutions x are called
the eigenvectors of the matrix A.the eigenvectors of the matrix A.

Solving for eigenvaluesSolving for eigenvalues
Ax=Ax=λλxx
Ax -Ax - λλx = 0x = 0
(A-(A- λλI)x = 0I)x = 0
 This is aThis is a homogeneoushomogeneous linear system,linear system,
homogeneous meaning that the RHS arehomogeneous meaning that the RHS are
all zeros.all zeros.
 For such a system, a theorem states thatFor such a system, a theorem states that
a solution exists given that det(A-a solution exists given that det(A- λλI)=0.I)=0.
 The eigenvalues are found by solving theThe eigenvalues are found by solving the
above equation.above equation.

Solving for eigenvalues cont’Solving for eigenvalues cont’
 Simple example: find the eigenvalues forSimple example: find the eigenvalues for
the matrix:the matrix:
 Eigenvalues are given by the equationEigenvalues are given by the equation
det(A-det(A-λλI) = 0:I) = 0:
 So, the roots of the last equation are -1So, the roots of the last equation are -1
and -6. These are theand -6. These are the eigenvalueseigenvalues ofof
matrix A.matrix A.






−
−
=
22
25
A
674)2)(5(
22
25
)det(
2
++=−−−−−=
−−
−−
=−
λλλλ
λ
λ
λ IA

EigenvectorsEigenvectors
 For each eigenvalue,For each eigenvalue, λλ, there is a, there is a
corresponding eigenvector, x.corresponding eigenvector, x.
 This vector can be found by substitutingThis vector can be found by substituting
one of the eigenvalues back into theone of the eigenvalues back into the
original equation: Ax =original equation: Ax = λλx : for thex : for the
example:example: -5x-5x11 + 2x+ 2x22 == λλxx11
2x2x11 – 2x– 2x22 == λλxx22
 UsingUsing λλ=-1, we get x=-1, we get x22 = 2x= 2x11, and by, and by
arbitrarily choosing xarbitrarily choosing x11 = 1, the eigenvector= 1, the eigenvector
corresponding tocorresponding to λλ=-1 is:=-1 is:
and similarly,and similarly,






=
2
1
1x 





−
=
1
2
2x

Linear algebra notes

More Related Content

What's hot

Similar to Linear algebra notes

More from Ghulam Mehdi Sahito

Recently uploaded

Linear algebra notes