Download SECTION B Properties of Eigenvalues and Eigenvectors

Chapter 7: Eigenvalues and Eigenvectors
13
SECTION B Properties of Eigenvalues and Eigenvectors
By the end of this section you will be able to
 determine eigenvalues and eigenvectors
 prove further properties of eigenvalues and eigenvectors
 apply the Cayley Hamilton Theorem
In the last section we found the eigenvectors of a matrix by solving simultaneous
equations using elimination. However this method can produce a messy and confused
way of solving simultaneous equations and a more structured approach is to use Gaussian
elimination or reduced row echelon form.
Eigenvalues and eigenvectors come in pairs. You cannot have an eigenvalue on its own;
it must have an associated eigenvector. They are pairs like chalk and cheese because
eigenvalue is a scalar and eigenvector is a vector. Do you know of any pairs outside the
world of mathematics?
Well known pairs are Caesar and Cleopatra, Romeo and Juliet, Oxford and Cambridge
etc.
First we look at multiple eigenvalues and their corresponding eigenvectors.
B1 Multiple Eigenvalues
What is the characteristic equation of a square matrix A?
The characteristic equation was defined in the last section A as
(7.2)
det  A  I   0
where A is a square n by n matrix, I is the identity matrix and  is the eigenvalue which
is a scalar. By expanding this we have
 a11
a1n    0


 

det  A   I   det 
0 
 0
 an1
ann   0
  
a1n 
 a11  
Taking Away  along 


 det 
 the leading diagonal 
0


 a

ann   
n1

and it can be shown that this is a polynomial equation of degree n.
Thus we nominate the characteristic polynomial p     det  A  I  and write it as
p         cn 1 n 1  cn  2  n  2 
n
 c2  2  c1  c0
where the c’s are the coefficients.
What is the difference between characteristic polynomial and equation?
Characteristic polynomial is an expression p    whilst an equation is p     0 .
How many roots does the equation p     0 have?
By the grand title of ‘Fundamental Theorem of Algebra’ we have that the polynomial
equation p     0 has exactly n roots. Don’t worry if you have not heard of the
‘Fundamental Theorem of Algebra’ but it is a known theorem in algebra which claims
that if there is a polynomial equation of degree n then it has exactly n roots. For example
x 2  2 x  1  0 has exactly two roots because it is a polynomial equation of degree 2
(quadratic equation). Other examples are:
Chapter 7: Eigenvalues and Eigenvectors
14
Equation
Number of Roots
5
5
x 1  0
12
2 x12  2 x3  1  0
101
5 x101  x100   1  0
We will not prove the Fundamental Theorem of Algebra but assume it is true.
Thus by the Fundamental Theorem of Algebra we conclude that
n
p         cn 1 n 1  cn  2 n 2   c2 2  c1  c0  0
has n eigenvalues (roots). They maybe n distinct eigenvalues or repeated such as
3
   1    2   0 gives 1  1, 2  1, 3  1 and 4  2
Normally we write the first 3 roots in compact form as 1, 2, 3  1 rather than as above.
We distinguish between simple and multiple eigenvalues in the next definition.
Definition (7.5). Let A be a n by n matrix and has the eigenvalues 1 , 2 , 3 ,
and
n . If  occurs only once then we say  is a simple eigenvalue otherwise it is called a
multiple eigenvalue. If  occurs m times where m  1 then we say  is an eigenvalue
with multiplicity of m or  has multiplicity m.
Example 7
 2 1 3
Determine the eigenvalues and eigenspaces of A   0 2 0  .
0 0 2


Solution
What do we determine first?
The eigenvalues because they produce the eigenvectors. Using the characteristic equation
(7.2)
det  A  I   0
2

det  A   I   det  0
 0

3 

2
0 
0
2   
0 
2
  2    det 

2
 0
1
  2     2    2     0    2     0
3
Thus we only have one repeated eigenvalue   1, 2, 3  2 . We say   2 has
multiplicity 3. How do we find the corresponding eigenvector?
By substituting this,   2 , into  A  I  u  O where u is the eigenvector:
1
3 
22

 A  2I  u   0 2  2 0  u  O
 0
0
2  2 

Simplifying this and substituting unknowns x, y, z for the eigenvector u and zeros into
the zero vector, O, gives
Chapter 7: Eigenvalues and Eigenvectors
15
 0 1 3  x   0 

   
 0 0 0  y    0 
 0 0 0  z   0 

   
Note that this matrix is already in reduced row echelon form (rref). There is only one
non-zero equation and 3 unknowns therefore there are 3 1  2 free variables (x and z
because they have no leading one). The term ‘free variables’ was defined in chapter 1.
From the first row of expansion we have y  3 z  0 which gives y  3 z . Let z  s
where s  0 then y  3s . Clearly x can be any real number, that is x  t . Hence we
write the eigenvector ( x  t , y  3s and z  s ) in terms of 2 separate vectors as shown
below:
 x   t   t   0   1  0
 s and t are 
  
   
    
our free 
u   y    3s    0    3s   t  0   s  3 


 z   s   0  s   0  1

variables
  
   
    


where s and t are not both zero. This u is our general eigenvector and we can write our
  1   0 
    
eigenspace as E2  t  0   s  3   and plot this in 3 as
  0   1 
    
 0
 
 3 
 1
 
E2
1
 
0
0
 
Fig 6
Note that we have a plane rather than just a line because we have 2 linearly independent
basis vectors (as shown above) which span a plane. A set of basis vectors B of the
eigenspace E2 are given by
 1   0  
    
B   0  ,  3  
 0   1  
    
The matrix given in the above example is a type of matrix called a triangular matrix
which was defined in Chapter 2. A smarter way to evaluate the eigenvalues of such
matrices is described next.
Chapter 7: Eigenvalues and Eigenvectors
16
B2 Eigenvalues of Diagonal and Triangular Matrices
In Exercise 7(a) you proved that the eigenvalues of the zero matrix, O, is 0. What sort
matrix is the zero matrix?
Diagonal matrix. What is a diagonal or triangular matrix?
By definition (2.17) of chapter 2 we have a triangular matrix is a n by n matrix where all
the entries to one side of the leading diagonal are zero.
By definition (2.18) of chapter 2 we have a diagonal matrix is a n by n matrix where
all the entries to both sides of the leading diagonal are zero.
The following are examples:
1 0 0 0 
 9 0 0 0 
 8 1 5 9 





0 8 3 6 
0 2 0 0
8 3
0 0 



A
, B
and C 
0 0 5 0
 2 4 1 0 
0 0 5 7 






5 10 
0 0 0 4 
 0 0 0 4
 1 0
Diagonal Matrix
Triangular Matrix
Triangular Matrix
B is actually called an upper triangular matrix and C a lower triangular matrix.
Next we prove that for a diagonal or triangular matrix the eigenvalues are given by the
entries along the leading diagonal.
Proposition (7.6). If a n by n matrix A is a diagonal or triangular matrix then the
eigenvalues of A are the entries along the leading diagonal.
How do we prove this?
We apply Proposition (2.19) from chapter 2 to the matrix A.
What does Proposition (2.19) claim?
Proposition (2.19) says  The determinant of a triangular or diagonal matrix is a product
of the entries along the leading diagonal.
Proof of (7.6).
a1n 
 a11 a12


0 a22
a2 n 
Without Loss of Generality let A  
be an upper triangular matrix.


0


0 ann 
 0
We prove it for this and the proof of lower triangular and diagonal matrix is very similar.
The characteristic equation det  A  I   0 is given by
a12
a1n 
 a11  


0
a22  
a2 n 
det  A   I   det 


0


0 ann   
 0

 a11    a22     ann     0
By Proposition (2.19)
and ann which are the
The roots or eigenvalues of this equation are a11 , a22 , a33 ,
leading diagonal entries of the given matrix A. Thus we have proven our result.
■
Chapter 7: Eigenvalues and Eigenvectors
17
Example 8
8
5 3 5


0 9 7 9 

Determine the eigenvalues of A 
.
 0 0 6 5 


 0 0 0 17 
Solution
What do you notice about matrix A?
Matrix A is a (upper) triangular matrix therefore we can apply the above Proposition
(7.6) to find the eigenvalues of matrix A. What are the eigenvalues of A?
The eigenvalues are the entries on the leading diagonal which runs from top left to
bottom right. Thus the eigenvalues are
1  5, 2  9, 3  6 and 4  17
Note that the eigenvalues of the matrix A given in Example 7 above are
1, 2, 3  2
because all the entries along the leading diagonal of the triangular matrix A are 2.
B3 Properties of Eigenvalues and Eigenvectors
Proposition (7.7). A square matrix A is invertible (non-singular)    0 is not an
eigenvalue of the matrix A.
Proof.
See Exercise 7b.
Proposition (7.8). Let A be a square matrix with eigenvector u belonging to eigenvalue  .
(a) If m is a natural number greater than 1  m  1 then  m is an eigenvalue of the matrix
Am with the same eigenvector u.
(b) If the matrix A is invertible (non-singular) then the eigenvalue of the inverse matrix
1
A 1 is   1 with the same eigenvector u.

What does this proposition mean?
Matrix
Eigenvector
Eigenvalue
A
u

m
u
(a)
A
m
u
(b)
A 1
 1
How do we prove proposition (a)?
By using mathematical induction. The procedure is to check the result for m  1 , assume
the result is true for m  k and then prove it for m  k  1.
Proof.
Using the definition of eigenvalues and eigenvectors (7.1) we have Au  u which
means the result holds for m  1 :
(*)
Au  u
Assume the result is true for m  k :
Ak u   k u
(†)
Required to prove the case m  k  1, that is we need to prove
A k 1u   k 1u
Expanding the Left Hand Side
Chapter 7: Eigenvalues and Eigenvectors
18
A k 1u  A  A k u 
 A   k u    k  Au    k  u    k  u   k 1u
By (*)
By  †
  k 1
Thus A k 1u   k 1u and so by mathematical induction we have our result that  m is an
eigenvalue of the matrix Am with the eigenvector u.
■
Proof of (b).
Using definition (7.1) we have Au  u . Multiplying both sides of this by A 1 gives
A 1A u  A 1  u    A 1u
I
Remember multiplying by the identity I keeps it the same, that is Iu  u so we have
u   A 1u
Dividing both sides by  gives
1
1
u  A 1u or writing this the other way we have A 1u  u


1
Again by the definition (7.1) we conclude that the eigenvalue of A 1 is
with the

eigenvector u.
■
Example 8
1 4 5
Find the eigenvalues of A 7 where A   0 2 6  .
 0 0 3


Solution
Because A is an upper triangular matrix so the eigenvalues are the entries on the leading
diagonal, that is 1  1,  2  2 and  3  3 . By the above Proposition (7.8) (a) we have
the eigenvalues of A 7 are
17  1, 27  128 and 37  2187
Proposition (7.9). Let A be a n by n matrix with eigenvalues 1 , 2 , 3 ,
and n .
(These eigenvalues may be repeated). We have
(a) The determinant of the matrix A is given by det  A   1  2  3   n .
(b) The trace of the matrix A is given by tr  A   1  2  3   n .
What does the first part (a) mean?
It means that we can find the determinant of the matrix by multiplying all the eigenvalues
of that matrix.
What does part (b) mean?
Adding all the eigenvalues of a matrix is equal to the trace of the matrix. What is the
trace of a matrix?
Trace of matrix is the addition of all the leading diagonal elements of the matrix. Can you
see any use for part (b)?
Chapter 7: Eigenvalues and Eigenvectors
19
We can use part (b) as a rough check to see if we have the correct eigenvalues. Why is (b)
a rough rather than an exact check for eigenvalues?
Because there are so many different ways of adding to the same value.
How do we prove part (a)?
By using the characteristic equation det  A  I   0 .
Proof of (a).
We are given that matrix A has eigenvalues 1 , 2 , 3 ,
and n . What does this
mean?
It means these, 1 , 2 , 3 ,
and n , are roots of the characteristic equation which
implies that we have
det  A  I    1    2    3     n   
[For example if the eigenvalues of a particular matrix are 1  1,  2  2 and  3  3 then
the characteristic equation would be given by det  A  I   1    2   3    .]
If we substitute   0 into this we get
det  A    1  0  2  0  3  0   n  0 
  1  2  3 
 n   123 n
This is our required result.
■
Proof of (b).
See Miscellaneous Exercise 7.
Example 9
1 2 1
Find the determinant and trace of A   2 1 1  given that the eigenvalues of A are 1, 4
1 1 2


and 1 .
Solution
By the above Proposition (7.9) we have
det  A   1 4   1  4 [Multiplying the eigenvalues]
Tr  A   1  4 1  4 [Adding all eigenvalues]
Remember the trace of the matrix is adding all the entries in the leading diagonal
Tr  A   1  1  2  4
Hence we have illustrated for matrix A that the trace of the matrix is the same as the
addition of all eigenvalues.
Proposition (7.10). Let A be a n by n matrix with distinct eigenvalues 1 , 2 , 3 ,
and
m and the corresponding eigenvectors u1 , u2 , u3 ,
and u m where 1  m  n . Then
and u m are linearly independent.
these eigenvectors u1 , u 2 , u3 ,
Proof.
See Exercise 7(b).
Chapter 7: Eigenvalues and Eigenvectors
20
B4 Cayley Hamilton Theorem
The biography of Arthur Cayley was given in Chapter 1 section B1. Here we give a brief
profile of Sir William Rowan Hamilton.
Fig 7 Hamilton 1805 to 1865
The formula was
Hamilton was born in Dublin, Ireland in 1805 and became one
of the greatest Irish mathematicians. Initially he took an
interest in languages but in his early school days he had
affection for mathematics. At the age of 18 he entered
Trinity College, Dublin and spent the rest of his life there.
In 1827 Hamilton was appointed Professor of Astronomy in
Trinity College but he did not take much interest in
astronomy and devoted all his time to mathematics. His
passion was mathematics.
Hamilton is best known for his work on quaternions which is
a vector space of 4 dimensions. In fact in 1843 whilst he
was walking along the local canal in Dublin with his wife he
had a flash of inspiration and discovered the formula for
quaternion multiplication.
i 2  j 2  k 2  ijk  1 and at present there is plaque at the bridge
stating this formula. He also invented the dot and cross product of vectors.
Throughout his life he had problems with alcohol and love. He fell in love with Catherine
but due to unfortunate circumstances he ended up marrying Helen which he regretted
for the rest of his life.
The statement of the Cayley Hamilton Theorem is straightforward.
Cayley Hamilton Theorem (7.11). Every square matrix A is a root of the characteristic
equation, that is p  A   O .
Proof.
This is a difficult proof and you are asked to show this in Miscellaneous Exercise 7.
Example 10
3 2
Find the characteristic polynomial p    of the matrix A  
 and illustrate the
3 4
Cayley Hamilton theorem for this matrix A.
Solution
We have p     det  A  I  and we substitute the given matrix A into this:
2 
3 
p     det  A   I   det 

4
 3
  3    4     6
Taking Away  Along 


 the leading Diagonal 
 12  7   2  6   2  7  6
Thus the characteristic polynomial is p      2  7  6 . What is p  A  equal to?
Chapter 7: Eigenvalues and Eigenvectors
21
p  A   A 2  7 A  6I
3

3
3

3
 15

 21
2
2
3 2 1 0
Substituting A and I 
  7
  6

4
3 4 0 1
2   3 2   21 14   6 0 
Carrying Out Scalar 






4   3 4   21 28   0 6 
 Multiplication

14   21 14   6 0 



22   21 28   0 6 
 15  21  6 14  14  0   0 0 


O
 21  21  0 22  28  6   0 0 
Thus p  A   O . What does this mean?
Means that matrix A satisfies its characteristic equation which is what the Cayley
Hamilton theorem states.
We can apply the Cayley Hamilton Theorem to find powers of matrices. For example if
 0.9 0.2 
 29.4 13.5 
100
A
 then A  
 (1dp)
 0.3 0.6 
 20.2 9.2 
A100 is not evaluated by multiplying 100 copies of matrix A. The evaluation of A100 is
found by using eigenvalues of matrix A.
Calculating A100 is a very tedious task which can be significantly reduced by applying
the Cayley Hamilton Theorem because the power of a matrix can be written in terms of a
polynomial of degree 1 in matrix A. This is illustrated in the example below.
Example 11
 2 4 
4
Let A  
 . Determine A .
 1 3
Solution
Working out the determinant of A  I gives
 2   4 
det  A   I   det 
   2    3     4
3 
 1
    2    3  4   2    2  p   
Hence p  A   A2  A  2I  O . Transposing this gives
 †
A 2  A  2I
 Multiplying by A 
A3  A 2  2A

A  2I
 2A  3A  2I
 A 2 by above  †
Multiplying the last equation A3  3A  2I by A gives
A 4  3A 2  2A
3
 A  2I 
A
2
 2A  5A  6I
by above  †
 2 4   1 0   4 20 
Thus A 4  5A  6I  5 
  6

.
21 
 1 3  0 1  5
Chapter 7: Eigenvalues and Eigenvectors
22
Note that we can evaluate A 4 without working out A3 and A 2 because A 4  5A  6I
which is polynomial of degree 1 in matrix A.
In MATLAB we can find the characteristic polynomial of a matrix A by using the
command poly(A).
We can also use the Cayley Hamilton Theorem to find the inverse of a matrix as the next
example demonstrates.
Example 12
 10 15 0 


Let A   2 4 0  and the characteristic polynomial of this matrix is given by
 3 6 6


p      3  20 2  94  60
Determine A 1 and A 4 in terms of the matrices A, I and A 2 .
Solution
By the Cayley Hamilton Theorem (7.11) we have
p  A   A3  20A2  94A  60I  O
Adding 60I to both sides we have
A3  20A 2  94A  60I
A  A 2  20A  94I   60I
 Factorizing Out the Matrix A 
By the definition of the inverse matrix we have AA 1  I which means dividing both
sides by 60 in the above gives
1
A  A 2  20A  94I   I
60
 A 1
1
A 2  20 A  94I  . How do we find A 4 ?

60
By rearranging the above A3  20A 2  94A  60I and making A3 the subject and then
multiplying by A:
A3  20A 2  94 A  60I
†
Hence A 1 
 
A 4  AA 3  A  20A 2  94A  60I 
 
 By † 


 20A3  94 A 2  60 A
 20  20 A 2  94 A  60I   94 A 2  60 A
 
 By † 


Replacing A3
 400A 2  1880A  1200I  94 A 2  60 A
 306A 2  1820A  1200I
 Expanding Brackets 
Collecting Like Terms 
Thus A 4  306A 2  1820A  1200I .
SUMMARY
If  occurs m times where m  1 then we say  is an eigenvalue with multiplicity of m.
Proposition (7.6). If A is a diagonal or triangular matrix then the eigenvalues of A are the
entries along the leading diagonal.
Let A be a square matrix with an eigenvalue  and eigenvector u belonging to  .
Chapter 7: Eigenvalues and Eigenvectors
23
Then we have the following:
Proposition (7.7). Matrix A is invertible if and only if   0 is not an eigenvalue.
Matrix
Eigenvector
Eigenvalue
m
u
(7.8) (a)
A
m
u
(7.8) (b)
A 1
 1
Proposition (7.9). If the eigenvalues of A are 1 , 2 , 3 ,
and n then we have:
(a) The determinant of the matrix A is given by det  A   1  2  3 
 n .
(b) The trace of the matrix A is given by tr  A   1  2  3   n .
Cayley Hamilton Theorem (7.11). Every square matrix A is a root of the characteristic
equation, that is p  A   O .