Download Chapter 7: Eigenvalues and Eigenvectors

Chapter 7: Eigenvalues and Eigenvectors
1
Chapter 7: Eigenvalues and Eigenvectors
SECTION A Introduction to Eigenvalues and Eigenvectors
By the end of this section you will be able to
 understand what is meant by an eigenvalue and eigenvector
 determine eigevalues and eigenvectors
 prove properties of eigenvalues and eigenvectors
This is an important chapter in linear algebra because it has many applications in the
areas of physical sciences and engineering. This section is straightforward but it does
rely on a number of topics in linear algebra such as matrices, determinants, vectors etc.
You need to thoroughly know how to evaluate determinants to understand this chapter.
A1 Definition of Eigenvalue and Eigenvector
Before we define what is meant by an eigenvalue and eigenvector we do an example
which involves these.
Example 1
 4 2 
 2
Let A  
 and u    then evaluate Au .
1 1 
1 
Solution.
Multiplying the matrix and column vector we have
 4 2  2   6 
 2
Au  
      3  
 1 1 1   3 
1 
What do you notice about your result?
Well we have Au  3u . In geometric terms we have that the matrix A scalar multiplies
the vector u by 3:
y
3
2.5
3
2
 
1
2
1.5
u=
1
 2
 
1
0.5
1
2
3
4
5
Fig 1
In general terms this can be written as
(7.1)
Au  u
where A is a square matrix, u is a column vector  u 
6
n
x
 and 
(lambda) is a scalar.
Can you think of a vector, u, which satisfies equation (7.1)?
The zero vector u  O . In this case we say we have the trivial solution. In this chapter
we consider the non-trivial solutions, u  O , and these solutions are powerful tools in
linear algebra.
For non-zero vector u the scalar  is called an eigenvalue of the matrix A and the
vector u is called an eigenvector belonging or corresponding to  which satisfies
Chapter 7: Eigenvalues and Eigenvectors
2
Au  u . In most linear algebra literature the Greek letter lambda,  , is used for
eigenvalues.
These terms eigenvalue and eigenvector are derived from the German word
‘eigenwerte’ which means ‘proper value.’ The word eigen is pronounced “i-gun”.
Eigenvalues was initially developed in the field of differential equations by Jean D’
Alembert.
Alembert 1717 to 1783 was a French
mathematician and the illegitimate son of
Madam Tencin and an army officer, Louis
Destouches. His mother left him on steps of
a local church but he was sent to a home for
homeless children. His father recognised his
son’s difficulties and placed him under the
care of Madam Rousseau. He always thought
of Rousseau as his mother.
Fig 2 Alembert 1717 to 1783
However Alembert’s father died when he was only nine years old but his father’s family
looked after his financial situation so that he could be educated.
In 1735 Alembert graduated and thought a career in law would suit him but his real thirst
and enthusiasm was for mathematics and did this in his spare time. Three years later he
did qualify for an advocate but did not pursue a career in this field but chose mathematics
instead.
For most of his life he worked for the Paris Academy of Science and the French Academy.
He was well known to have a short fuse and loved to argue with most of his
contemporaries.
Example 2
 1 1
Let A  
 . Verify the following:
 2 4 
1
(a) u    is an eigenvector of the matrix A corresponding to the eigenvalue 1  2 .
1
1 
(b) v    is an eigenvector of the matrix A corresponding to the eigenvalue 2  3 .
 2
Solution.
(a) Multiplying the given matrix A and column vector u we have
 1 1 1  2 
 1
Au  
      2  
 2 4 1  2 
 1
1
Thus u    is an eigenvector of the matrix A belonging to 1  2 because Au  2u .
1
(b) Similarly we have
 1 1 1   3 
1 
Av  
      3  
 2 4  2   6 
 2
1 
Thus v    is an eigenvector of the matrix A belonging to 2  3 because Av  3v .
 2
Chapter 7: Eigenvalues and Eigenvectors
3
What do you notice about your results?
A 2 by 2 matrix can have more than one eigenvalue and eigenvector.
Note that we can have eigenvalues and eigenvectors for any square matrices such as
3  3 , 4  4 , 5  5 , etc which satisfy Au  u .
Example 3
5 0 0
0


Let A   9 4 1 and u  1  . Determine an eigenvalue of this matrix A.
 2
 6 2 1 
 


Solution.
Multiplying the matrix and column vector we have
 5 0 0  0   0 
0

   
 
Au   9 4 11    2   2  1 
 6 2 1  2   4 
 2

   
 
We have Au  u where   2 . Hence   2 is an eigenvalue of the matrix A with an
eigenvector u.
A2 Characteristic Equation
From the above equation (7.1) Au  u we have
Au  I nu [ I nu  u Multiplying by the identity keeps it the same]
where I n is the n by n identity matrix. We can rewrite this as
Au  I nu  O
 A  I n  u  O
Under what condition is the non-zero vector u a solution of this equation?
By Proposition (2.30)?? of chapter 2 we have a non-zero vector u if and only if
det  A  I n   0
This is an important equation and is called the characteristic equation. We give this a
reference number
(7.2)
det  A  I   0
We drop the subscript n because we assume the identity matrix I is of appropriate size.
In some linear algebra texts you will see (7.2) written as det  I  A   0 . Both of these
are equivalent so it does not matter which we apply to find the eigenvalues.
This second approach does have the advantage of not expanding brackets like
 1    2   
But if you remove 2 minus signs then it should easier to expand this:
 1    2       1    2     1    2   

Advantage of using (7.2) is that you don’t need to subtract the matrix A, it just boils
down to subtracting  along the leading diagonal.
We use this characteristic equation to find the eigenvalues and corresponding
eigenvectors. The procedure for this is as follows:
1. Solve the characteristic equation for the scalar  .
2. For the eigenvalue  determine the corresponding eigenvector u by solving the
system  A  I  u  O .
Chapter 7: Eigenvalues and Eigenvectors
4
Let’s follow this procedure for the next example.
Note that eigenvalues and eigenvectors come in pairs. You cannot have one without
the other.
Example 4
 2 0
Determine the eigenvalues and corresponding eigenvectors of A  
.
 1 3
Also sketch the effect of multiplying the eigenvectors by matrix A on 2 .
Solution.
What do we find first, the eigenvalues or eigenvectors?
Eigenvalues because eigenvectors are born out of eigenvalues. Remember we carry out
the procedure outlined above:
Step 1. Solve the characteristic equation for the scalar  .
We need to find λ in det  A  I  . First we obtain A  I :
2
A  I  
1
2

1
0
1 0


3
0 1
0  0   2  
0 



3  0    1
3 
Substituting this into det  A  I  gives
0 
2
det  A   I   det 
   2    3     0
3 
 1
For eigenvalues we equate this determinant to zero:
 2    3     0
1  2 or 2  3
Step 2. For the eigenvalue  determine the corresponding eigenvector u by solving the
system  A  I  u  O .
 2 0
Let u be the eigenvector corresponding to 1  2 . Substituting A  
 and
 1 3
1    2 into  A  I  u  O gives
 2 0   2 0  

 u  O
 1 3   0 2  
0 0

u  O
1 1
 A   I  u  
x
0
Remember O    and let u    and so we have
 y
0
 0 0  x   0 

    
 1 1  y   0 
Multiplying out gives
00  0
x y 0
Remember the eigenvector cannot be the zero vector, therefore at least one of the
values, x or y, must be non-zero. From the bottom equation we have x   y .
Chapter 7: Eigenvalues and Eigenvectors
5
 2   3      5 
Simplest solution is x  1, y  1 but we can have   ,   ,   ,   etc
 2   3      5 
We have an infinite number of solutions. We need to write down the general
eigenvector u. How?
Let x  s then y   s where s  0 and is a scalar. Thus the eigenvectors belonging to
  2 are
 1

 s
 1
u     s   where s  0
  is the simplest eigenvector 
 s 
 1
 1 

Similarly we find the general eigenvector v corresponding to the other eigenvalue
2  3 . Putting 2    3 into  A  I  v  O gives
 2 0   3 0  

  v  O simplifies to
1
3
0
3
 


 1 0 

v O
1
0


0
[different x and y from those above] and O    we obtain
0
 1 0  x   0 

    
 1 0  y   0 
 A   I  v  
x
By writing v   
 y
Multiplying out:
 x  0  0, x  0  0
From these equations we must have x  0 . What is y equal to?
We can choose y to be any real number apart from zero. Thus
y  s where s  0
The general eigenvector belonging to 2  3 is
 0 

 x  0
 0
v        s   where s  0   is the simplest eigenvector 
 y s 
1 
1 

Summarizing the above we have for the eigenvalue 1  2 the eigenvector
 1
0
u  s   and for 2  3 the eigenvector v  s   . What does all this mean?
 1
1 
It means that we must have Au  1u and Av  2 v which in this case is
Au  2u and Av  3v
Plotting these eigenvectors and the effect of multiplying by the matrix A is shown:
y
3
Av = 3v
u and v shown are
the simplest
eigenvectors.
2
1
v=
 0
 
1
0.5
-1
Fig 3
-2
u=
1
1
 
 -1 
1.5
2
Au = 2u
x
Av = 3v
Chapter 7: Eigenvalues and Eigenvectors
6
The effect on the eigenvectors of multiplying by the matrix A is to produce scalar
multiples of itself as you can see in Fig 3.
Eigenvectors are non-zero vectors which are transformed to scalar multiples of
themselves by a square matrix A.
Next we find the eigenvalues and eigenvectors of a 3 by 3 matrix. Follow the algebra
carefully because you will have to expand brackets like
1    3   
To expand this it is easier to take out 2 minus signs and then expand, that is
1    3        1  3       13    Because    
Example 5
1 0 4 
Determine the eigenvalues of A   0 4 0 
 3 5 3 


Solution.
We have
0
4 
 1 0 4    0 0  1  

 
 

A  I   0 4 0    0  0    0
4
0 
 3 5 3   0 0    3
5
3   

 
 
It is easier to remember that A  I is actually matrix A with  along the leading
diagonal (from top left to bottom right). We need to evaluate det  A  I  .
What is the easiest way to find this?
From the properties of determinants of chapter 2 we know that it will be easier to
evaluate the determinant along the second row, containing the elements 0, 4   and 0.
Why?
Because it has two zeros so that we do not have to evaluate the 2 by 2 determinants
associated with these zeros. From above we have
0
4 
1  


det  A   I   det  0
4
0 
Row 2
 3
5
3   

 1  
4 
  4    det 

3    
  3
  4    1    3      3  4  
 Expanding the 
Second Row 


 By Determinant of 2 by 2
  4       1 3     12 
Taking Out Minus Signs 
  4    3   2  3    12 
Opening Brackets and     
  4     2  2  15
Simplifying 
  4      5    3
 Factorizing 
By the characteristic equation (7.2) which is det  A  I   0 , means that we equate all
the above to zero:
 4      5   3  0
Chapter 7: Eigenvalues and Eigenvectors
7
Solving this gives the eigenvalues 1  4, 2  5 and 3  3 .
Example 6
Determine the eigenvectors associated with 3  3 for the matrix A given in the above
Example 5.
Solution.
1 0 4 
Substituting the eigenvalue 3    3 and the matrix A   0 4 0  into
 3 5 3 


 A  I  u  O (subtract 3 from the leading diagonal) gives:
0
4 
1  3

 A  3I  u   0 4  3 0  u  O
 3
5
3  3 

where u is the eigenvector corresponding to 3  3 .
What is the zero vector, O, equal to?
0
x


Remember this zero vector is O   0  and also let u   y  . Substituting these into the
0
z 
 
 
above and simplifying gives
 2 0 4  x   0 

   
 0 1 0  y    0 
 3 5 6  z   0 

   
Expanding this yields the linear system
2 x  0  4 z  0
 †
0 y0  0
3x  5 y  6 z  0
 ††
 †††
From the middle equation  †† we have y  0 . From the top equation  † we have
2x  4z which gives x  2z
If z  1 then x  2 ; or more generally if z  s then x  2s where s  0 [Not Zero].
 x   2s 
 2




 
The general eigenvector u   y    0   s  0  where s  0 and corresponds to
z   s
1 
   
 
3  3 .
The eigenvectors corresponding to 1  4 and 2  5 are part of Exercise 7a.
A3 Eigenspace
Note that for the 3  3 in the above Example 6 we have an infinite number of
eigenvectors by substituting various non-zero values of s:
Chapter 7: Eigenvalues and Eigenvectors
8
 2
 4
1 
 4 
 
 


 
u   0  or u   0  or u   0  or u   0  etc
1 
 2
1/ 2 
 2 
 
 


 
These solutions are given by all the points (apart from x  y  z  0 ) on the line shown
below:
 4
 
0
 2
 
 4 
 
 0
 2 
 
Fig 4
In general if A is a square matrix and  is an eigenvalue of A with an eigenvector u
then every scalar multiply (apart from 0) of the vector u is also an eigenvector
belonging to the eigenvalue  . We write this as a proposition:
Proposition (7.3). If  is an eigenvalue of a square matrix A with an eigenvector u then
every non-zero scalar multiplication of u is also an eigenvector belonging to  .
Proof.
By (7.1) we have Au  u where  is the eigenvalue and u is the eigenvector
belonging to  . Consider an arbitrary non-zero scalar k, then
A  k u   k  Au 
 By rules of Matrices
 k  u 
  ku
 By  7.1 
Thus we have A  k u     k u  which means that k u is an eigenvector belonging to the
eigenvalue  . Since k was arbitrary therefore every non-zero scalar multiple of u is an
eigenvector of the matrix A belonging to the eigenvalue  .
■
Proposition (7.4). If A is a square n  n matrix with an eigenvalue of  , then the set of
all eigenvectors of A belonging to  together with the zero vector, O, that is the
following set
S  O  u u is an eigenvector belonging to 
is a subspace of n .
How do we prove the given set is a subspace of
n
?
Chapter 7: Eigenvalues and Eigenvectors
9
We can use proposition (4.7) of chapter 4 which says that any non-empty subset S of a
vector space is a subspace if and only if
(a)
The zero vector, O, is in S
(b)
If vectors u and v are in S then for any scalars k and c we have k u  c v is also
in S.
Proof.
What do we need to prove?
We need to prove the zero vector, O, is in the set S and if u and v are eigenvectors
belonging to the eigenvalue  then k u  c v is also an eigenvector belonging to  .
Clearly by the definition of the set S we have the zero vector in S.
Let u and v be eigenvectors belonging to the eigenvalue  and k and c be any non-zero
scalars. Then by the above Proposition (7.3) we have
A  k u     k u  and A  c v     c v 
We need to show that A  k u  c v     k u  c v  :
 Applying the rules of matrices 
 By the above
A  ku  cv  A ku  A cv
  ku   cv
  ku  cv
Thus k u  c v is an eigenvector belonging to  which means it is a member of the set
S. By Proposition (4.7) we conclude that the set S is a subspace of n .
■
This subspace S of Proposition (7.4)
S  O  u u is an eigenvector belonging to 
is called an eigenspace of λ and is denoted by E , that is E  S .
For example the eigenspace relating to Example 4 for the eigenvalue 1  2 is the
 1
0
eigenvector u  s   and for 2  3 the eigenvector v  s   is shown below:
 1
1 
y
3
E3
Vectors u and v
shown are the
simplest
eigenvectors.
2
1
-3
-2
v=
-1
1
-1
u=
-2
Fig 5
 0
 
1
-3
1
 
 -1 
2
3
E2
x
Chapter 7: Eigenvalues and Eigenvectors
10
0
 1
Note that the vector   is a basis for the eigenspace E3 and   is a basis for the
1 
 1 
eigenspace E2 . The simplest eigenvectors are a basis for each eigenspace.
The line in Fig 4 with arrows pointing to it is the eigenspace E3 of Example 5.
We can also use the numerical software MATLAB to find eigenvalues and eigenvectors
as the following commands verify:
 1 2 3
Let A   4 5 6  and enter this matrix in MATLAB. The command eig(A) gives you
7 8 9


the eigenvalues of this matrix which is 16.1168,  1.1168 and 0 . The following
command
>> [V,d]=eig(A) should show
V=
Reading each column down gives
0.2320  0.7858 0.4082
the eigenvectors and they have been
0.5253  0.0868 0.8165
normalized.
0.8187 0.6123 0.4082
d=
The diagonal entries give the
16.1168
0
0
eigenvalues
0 1.1168
0
0
0 0.0000
MATLAB gives the normalized eigenvectors. Note that we can normalize our
eigenvectors by dividing by the length or norm of the vector.
By reading the above MATLAB output we have the normalized eigenvectors are
 0.2320 
 0.7858 
 0.4082 

,

 and


w   0.8165 
u   0.5253  v   0.0868 
 0.4082 
 0.8187 
 0.6123 






SUMMARY
Consider
(7.1)
Au  u
where A is a square matrix, u is a column vector and  is a scalar. If u is non-zero
vector then we call u an eigenvector belonging (or corresponding) to the eigenvalue 
of the matrix A.
The following equation
(7.2)
det  A  I   0
is called the characteristic equation.
To find the eigenvalues and eigenvectors we use the following procedure:
1. Solve the characteristic equation for the scalar  .
2. For the eigenvalue  determine the corresponding eigenvector u by solving the
system  A  I  u  O .
Proposition (7.3). If  is an eigenvalue of a square matrix A with an eigenvector u then
every non-zero scalar multiplication of u is also an eigenvector belonging to  .
Proposition (7.4). If A is a square n  n matrix with an eigenvalue of  , then the
following set
E  O  u u is an eigenvector belonging to 
is a subspace of
n
and is called the eigenspace.