Theory and analysis of eigen values and eigen vectors
TARUN GEHLOT B.E (CIVIL, HONOURS)THEORY AND ANALYSIS OF EIGEN VALUES AND EIGEN VECTORSThe Eigen value problem is a problem of considerable theoretical interest andwide-ranging application. For example, this problem is crucial in solving systemsof differential equations, analyzing population growth models, and calculatingpowers of matrices (in order to define the exponential matrix). Other areas suchas physics, sociology, biology, economics and statistics have focusedconsiderable attention on "Eigen values" and "eigenvectors"-their applicationsand their computations. Before we give the formal definition, let us introducethese concepts on an example.Example. Consider the matrixConsider the three column matricesWe haveIn other words, we haveNext consider the matrix P for which the columns are C1, C2, and C3, i.e.,
TARUN GEHLOT B.E (CIVIL, HONOURS)We have det(P) = 84. So this matrix is invertible. Easy calculations giveNext we evaluate the matrix P-1AP. We leave the details to the reader to checkthat we haveIn other words, we haveUsing the matrix multiplication, we obtainwhich implies that A is similar to a diagonal matrix. In particular, we have
TARUN GEHLOT B.E (CIVIL, HONOURS)for . Note that it is almost impossible to find A75directly from theoriginal form of A.This example is so rich of conclusions that many questions impose themselves ina natural way. For example, given a square matrix A, how do we find columnmatrices which have similar behaviors as the above ones? In other words, howdo we find these column matrices which will help find the invertible matrix P suchthat P-1AP is a diagonal matrix?From now on, we will call column matrices vectors. So the above columnmatrices C1, C2, and C3 are now vectors. We have the following definition.Definition . Let A be a square matrix. A non-zero vector C is calledan eigenvector of A if and only if there exists a number (real or complex)such thatIf such a number exists, it is called an eigenvalue of A. The vector C is calledeigenvector associated to the Eigen value .Remark. The eigenvector C must be non-zero since we havefor any number .Example. Consider the matrixWe have seen that
TARUN GEHLOT B.E (CIVIL, HONOURS)whereSo C1 is an eigenvector of A associated to the eigenvalue 0. C2 is an eigenvectorof A associated to the eigenvalue -4 while C3 is an eigenvector of A associated tothe eigenvalue 3.It may be interesting to know whether we found all the eigenvalues of A in theabove example. In the next page, we will discuss this question as well as how tofind the eigenvalues of a square matrix.For a square matrix A of order n, the number is an eigenvalue if and only ifthere exists a non-zero vector C such thatUsing the matrix multiplication properties, we obtainThis is a linear system for which the matrix coefficient is . We also knowthat this system has one solution if and only if the matrix coefficient is invertible,i.e. . Since the zero-vector is a solution and C is not the zerovector, then we must have
TARUN GEHLOT B.E (CIVIL, HONOURS)Example. Consider the matrixThe equation translates intowhich is equivalent to the quadratic equationSolving this equation leads toIn other words, the matrix A has only two eigenvalues.In general, for a square matrix A of order n, the equationwill give the eigenvalues of A. This equation is called the characteristicequation or characteristic polynomial of A. It is a polynomial function in ofdegree n. So we know that this equation will not have more than n roots orsolutions. So a square matrix A of order n will not have more than n eigenvalues.Example. Consider the diagonal matrix
TARUN GEHLOT B.E (CIVIL, HONOURS)Its characteristic polynomial isSo the eigenvalues of D are a, b, c, and d, i.e. the entries on the diagonal.This result is valid for any diagonal matrix of any size. So depending on thevalues you have on the diagonal, you may have one eigenvalue, twoeigenvalues, or more. Anything is possible.Remark. It is quite amazing to see that any square matrix A has the sameeigenvalues as its transpose ATbecauseFor any square matrix of order 2, A, wherethe characteristic polynomial is given by the equation
TARUN GEHLOT B.E (CIVIL, HONOURS)The number (a+d) is called the trace of A (denoted tr(A)), and clearly the number(ad-bc) is the determinant of A. So the characteristic polynomial of A can berewritten asLet us evaluate the matrixB = A2- tr(A) A + det(A) I2.We haveWe leave the details to the reader to check thatIn other word, we haveThis equation is known as the Cayley-Hamilton theorem. It is true for anysquare matrix A of any order, i.e.
TARUN GEHLOT B.E (CIVIL, HONOURS)where is the characteristic polynomial of A.We have some properties of the eigenvalues of a matrix.Theorem. Let A be a square matrix of order n. If is an eigenvalue of A, then:1.is an eigenvalue of Am, for2.If A is invertible, then is an eigenvalue of A-1.3.A is not invertible if and only if is an eigenvalue of A.4.If is any number, then is an eigenvalue of .5.If A and B are similar, then they have the same characteristic polynomial(which implies they also have the same eigenvalues).Let A be a square matrix of order n and one of its eigenvalues. Let X be aneigenvector of A associated to . We must haveThis is a linear system for which the matrix coefficient is . Since thezero-vector is a solution, the system is consistent. In fact, we will in a differentpage that the structure of the solution set of this system is very rich. In this page,we will basically discuss how to find the solutions.Remark. It is quite easy to notice that if X is a vector which satisfies ,then the vector Y = c X (for any arbitrary number c) satisfies the same equation,i.e. . In other words, if we know that X is an eigenvector, then cX isalso an eigenvector associated to the same eigenvalue.
TARUN GEHLOT B.E (CIVIL, HONOURS)Let us start with an example.Example. Consider the matrixFirst we look for the eigenvalues of A. These are given by the characteristicequation , i.e.If we develop this determinant using the third column, we obtainUsing easy algebraic manipulations, we getwhich implies that the eigenvalues of A are 0, -4, and 3.Next we look for the eigenvectors.1.Case : The associated eigenvectors are given by the linear system
TARUN GEHLOT B.E (CIVIL, HONOURS)which may be rewritten byMany ways may be used to solve this system. The third equation isidentical to the first. Since, from the second equations, we have y = 6x, thefirst equation reduces to 13x + z = 0. So this system is equivalent toSo the unknown vector X is given byTherefore, any eigenvector X of A associated to the eigenvalue 0 is givenbywhere c is an arbitrary number.2.Case : The associated eigenvectors are given by the linearsystem
TARUN GEHLOT B.E (CIVIL, HONOURS)which may be rewritten byIn this case, we will use elementary operations to solve it. First weconsider the augmented matrix , i.e.Then we use elementary row operations to reduce it to a upper-triangularform. First we interchange the first row with the first one to getNext, we use the first row to eliminate the 5 and 6 on the first column. WeobtainIf we cancel the 8 and 9 from the second and third row, we obtain
TARUN GEHLOT B.E (CIVIL, HONOURS)Finally, we subtract the second row from the third to getNext, we set z = c. From the second row, we get y = 2z = 2c. The first rowwill imply x = -2y+3z = -c. HenceTherefore, any eigenvector X of A associated to the eigenvalue -4 is givenbywhere c is an arbitrary number.2.Case : The details for this case will be left to the reader. Usingsimilar ideas as the one described above, one may easily show that anyeigenvector X of A associated to the eigenvalue 3 is given by
TARUN GEHLOT B.E (CIVIL, HONOURS)where c is an arbitrary number.Remark. In general, the eigenvalues of a matrix are not all distinct from eachother (see the page on the eigenvalues for more details). In the next twoexamples, we discuss this problem.Example. Consider the matrixThe characteristic equation of A is given byHence the eigenvalues of A are -1 and 8. For the eigenvalue 8, it is easy to showthat any eigenvector X is given bywhere c is an arbitrary number. Let us focus on the eigenvalue -1. Theassociated eigenvectors are given by the linear system
TARUN GEHLOT B.E (CIVIL, HONOURS)which may be rewritten byClearly, the third equation is identical to the first one which is also a multiple ofthe second equation. In other words, this system is equivalent to the systemreduced to one equation2x+y + 2z= 0.To solve it, we need to fix two of the unknowns and deduce the third one. Forexample, if we set and , we obtain . Therefore, anyeigenvector X of A associated to the eigenvalue -1 is given byIn other words, any eigenvector X of A associated to the eigenvalue -1 is a linearcombination of the two eigenvectorsExample. Consider the matrix
TARUN GEHLOT B.E (CIVIL, HONOURS)The characteristic equation is given byHence the matrix A has one eigenvalue, i.e. -3. Let us find the associatedeigenvectors. These are given by the linear systemwhich may be rewritten byThis system is equivalent to the one equation-systemx - y = 0.So if we set x = c, then any eigenvector X of A associated to the eigenvalue -3 isgiven by
TARUN GEHLOT B.E (CIVIL, HONOURS)Let us summarize what we did in the above examples.Summary: Let A be a square matrix. Assume is an eigenvalue of A. In orderto find the associated eigenvectors, we do the following steps:1.Write down the associated linear system2.Solve the system.3.Rewrite the unknown vector X as a linear combination of known vectors.The above examples assume that the eigenvalue is real number. So one maywonder whether any eigenvalue is always real. In general, this is not the caseexcept for symmetric matrices. The proof of this is very complicated. For squarematrices of order 2, the proof is quite easy. Let us give it here for the sake ofbeing little complete.Consider the symmetric square matrixIts characteristic equation is given byThis is a quadratic equation. The nature of its roots (which are the eigenvaluesof A) depends on the sign of the discriminant
TARUN GEHLOT B.E (CIVIL, HONOURS)Using algebraic manipulations, we getTherefore, is a positive number which implies that the eigenvalues of A arereal numbers.Remark. Note that the matrix A will have one eigenvalue, i.e. one double root, ifand only if . But this is possible only if a=c and b=0. In other words, wehaveA = a I2.First let us convince ourselves that there exist matrices with complexeigenvalues.Example. Consider the matrixThe characteristic equation is given byThis quadratic equation has complex roots given by
TARUN GEHLOT B.E (CIVIL, HONOURS)Therefore the matrix A has only complex eigenvalues.The trick is to treat the complex eigenvalue as a real one. Meaning we deal withit as a number and do the normal calculations for the eigenvectors. Let us seehow it works on the above example.We will do the calculations for . The associated eigenvectors aregiven by the linear systemA X = (1+2i) Xwhich may be rewritten asIn fact the two equations are identical since (2+2i)(2-2i) = 8. So the systemreduces to one equation(1-i)x - y = 0.Set x=c, then y = (1-i)c. Therefore, we havewhere c is an arbitrary number.Remark. It is clear that one should expect to have complex entries in theeigenvectors.We have seen that (1-2i) is also an eigenvalue of the above matrix. Since theentries of the matrix A are real, then one may easily show that if is a complex
TARUN GEHLOT B.E (CIVIL, HONOURS)eigenvalue, then its conjugate is also an eigenvalue. Moreover, if X is aneigenvector of A associated to , then the vector , obtained from X by takingthe complex-conjugate of the entries of X, is an eigenvector associated to . Sothe eigenvectors of the above matrixA associated to the eigenvalue (1-2i) aregiven bywhere c is an arbitrary number.Let us summarize what we did in the above example.Summary: Let A be a square matrix. Assume is a complex eigenvalue of A. Inorder to find the associated eigenvectors, we do the following steps:1.Write down the associated linear system2.Solve the system. The entries of X will be complex numbers.3.Rewrite the unknown vector X as a linear combination of known vectorswith complex entries.
TARUN GEHLOT B.E (CIVIL, HONOURS)4.If A has real entries, then the conjugate is also an eigenvalue. Theassociated eigenvectors are given by the same equation found in 3, exceptthat we should take the conjugate of the entries of the vectors involved inthe linear combination.In general, it is normal to expect that a square matrix with real entries may stillhave complex eigenvalues. One may wonder if there exists a class of matriceswith only real eigenvalues. This is the case for symmetric matrices. The proof isvery technical and will be discussed in another page. But for square matrices oforder 2, the proof is quite easy. Let us give it here for the sake of being littlecomplete.Consider the symmetric square matrixIts characteristic equation is given byThis is a quadratic equation. The nature of its roots (which are the eigenvaluesof A) depends on the sign of the discriminantUsing algebraic manipulations, we get
TARUN GEHLOT B.E (CIVIL, HONOURS)Therefore, delta is a positive number which implies that the eigen valuesof A are real numbers.When we introduced eigenvalues and eigenvectors, we wondered when a squarematrix is similarly equivalent to a diagonal matrix? In other words, given a squarematrix A, does a diagonal matrix D exist such that ? (i.e. there exists aninvertible matrix P such that A = P-1DP)In general, some matrices are not similar to diagonal matrices. For example,consider the matrixAssume there exists a diagonal matrix D such that A = P-1DP. Then we havei.e is similar to . So they have the same characteristicequation. Hence A and D have the same eigenvalues. Since the eigenvaluesof D of the numbers on the diagonal, and the only eigenvalue of A is 2, then wemust haveIn this case, we must have A = P-1DP = 2 I2, which is not the case.Therefore, A is not similar to a diagonal matrix.Definition. A matrix is diagonalizable if it is similar to a diagonal matrix.
TARUN GEHLOT B.E (CIVIL, HONOURS)Remark. In a previous page, we have seen that the matrixhas three different eigenvalues. We also showed that A is diagonalizable. In fact,there is a general result along these lines.Theorem. Let A be a square matrix of order n. Assume that A has n distincteigenvalues. Then A is diagonalizable. Moreover, if P is the matrix with thecolumns C1, C2, ..., and Cn the n eigenvectors of A, then the matrixP-1AP is adiagonal matrix. In other words, the matrix A is diagonalizable.Problem: What happened to square matrices of order n with less than neigenvalues?We have a partial answer to this problem.Theorem. Let A be a square matrix of order n. In order to find out whether A isdiagonalizable, we do the following steps:1.Write down the characteristic polynomial2.Factorize . In this step, we should be able to get
TARUN GEHLOT B.E (CIVIL, HONOURS)where the , , may be real or complex. For every i, thepowers ni is called the (algebraic) multiplicity of the eigenvalue .3.For every eigenvalue, find the associated eigenvectors. For example, forthe eigenvalue , the eigenvectors are given by the linear systemThen solve it. We should find the unknown vector X as a linearcombination of vectors, i.e.where , are arbitrary numbers. The integer mi is calledthe geometric multiplicity of .4.If for every eigenvalue the algebraic multiplicity is equal to the geometricmultiplicity, then we havewhich implies that if we put the eigenvectors Cj, we obtained in 3. for all theeigenvalues, we get exactly n vectors. Set P to be the square matrix oforder n for which the column vectors are the eigenvectors Cj. ThenP isinvertible and
TARUN GEHLOT B.E (CIVIL, HONOURS)is a diagonal matrix with diagonal entries equal to the eigenvalues of A.The position of the vectors Cj in P is identical to the position of theassociated eigenvalue on the diagonal of D. This identity implies that A issimilar to D. Therefore, A is diagonalizable.Remark. If the algebraic multiplicity ni of the eigenvalue is equal to 1,then obviously we have mi = 1. In other words, ni = mi.5.If for some eigenvalue the algebraic multiplicity is not equal to thegeometric multiplicity, then A is not diagonalizable.Example. Consider the matrixIn order to find out whether A is diagonalizable, lt us follow the steps describedabove.1.The polynomial characteristic of A isSo -1 is an eigenvalue with multiplicity 2 and -2 with multiplicity 1.2.
TARUN GEHLOT B.E (CIVIL, HONOURS)In order to find out whether A is diagonalizable, we only concentrate urattention on the eigenvalue -1. Indeed, the eigenvectors associated to -1,are given by the systemThis system reduces to the equation -y + z = 0. Set and ,then we haveSo the geometric multiplicity of -1 is 2 the same as its algebraic multiplicity.Therefore, the matrix A is diagonalizable. In order to find the matrix P weneed to find an eigenvector associated to -2. The associated system iswhich reduces to the systemSet , then we have
TARUN GEHLOT B.E (CIVIL, HONOURS)SetThenBut if we setthenWe have seen that if A and B are similar, then Ancan be expressed easily interms of Bn. Indeed, if we have A = P-1BP, then we have An= P-1BnP. Inparticular, if D is a diagonal matrix, Dnis easy to evaluate. This is one application
TARUN GEHLOT B.E (CIVIL, HONOURS)of the diagonalization. In fact, the above procedure may be used to find thesquare root and cubic root of a matrix. Indeed, consider the matrix aboveSetthenHence A = P D P-1. SetThen we haveB3= A.In other words, B is a cubic root of A.