Download Eigenvalues

Solving Linear Systems
• Solving linear systems Ax = b is one part of
•
•
numerical linear algebra, and involves
manipulating the rows of a matrix.
The second main part of numerical linear
algebra is about transforming a matrix to leave
its eigenvalues unchanged.
Ax = x
where  is an eigenvalue of A and non-zero x is
the corresponding eigenvector.
The symbol  is the Greek symbol for Lambda
What are Eigenvalues?
• Eigenvalues are important in physical,
biological, and financial problems (and
others) that can be represented by
ordinary differential equations.
• Eigenvalues often represent things like
natural frequencies of vibration, energy
levels in quantum mechanical systems,
stability parameters.
What are Eigenvectors
• Mathematically speaking, the eigenvectors of
•
•
•
matrix A are those vectors that when multiplied
by A are parallel to themselves.
Finding the eigenvalues and eigenvectors is
equivalent to transforming the underlying
system of equations into a special set of
coordinate axes in which the matrix is diagonal.
The eigenvalues are the entries of the diagonal
matrix.
The eigenvectors are the new set of coordinate
axes.
Determinants
• Suppose we if eliminate the components of x
•
•
•
•
from Ax=0 using Gaussian Elimination without
pivoting.
We do the kth forward elimination step by
subtracting ajk times row k from akk times row j,
for j=k,k+1,…n.
Then at the end of forward elimination we have
Ux=0, and Unn is the determinant of A, det(A).
For nonzero solutions to Ax=0 we must have
det(A) = 0.
Determinants are defined only for square
matrices.
Determinant Example
 a11 a12

• Suppose we have a
A   a21 a22
3x3 matrix.
a
• So Ax=0 is the same as:
 31 a32
a11x1+a12x2+a13x3 = 0
a21x1+a22x2+a23x3 = 0
a31x1+a32x2+a33x3 = 0
a13 

a23 
a33 
Determinant Example (continued)
• Step k=1:
– subtract a21 times equation 1 from a11 times
equation 2.
– subtract a31 times equation 1 from a11 times
equation 3.
• So we have:
(a11a22-a21a12)x2+(a11a23-a21a13)x3 = 0
(a11a32-a31a12)x2+(a11a33-a31a13)x3 = 0
Determinant Example (continued)
• Step k=2:
•
– subtract (a11a32-a31a12) times equation 2 from (a11a22a21a12) times equation 3.
So we have:
[(a11a22-a21a12)(a11a33-a31a13)- (a11a32-a31a12)(a11a23-a21a13)]x3 = 0
which becomes:
[a11(a22a33 –a23a32) – a12(a21a33-a23a31) + a13(a21a32-a22a31)]x3 = 0
and so:
det(A) = a11(a22a33 –a23a32) – a12(a21a33-a23a31) + a13(a21a32-a22a31)
Definitions
• Minor Mij of matrix A is the determinant of the
•
•
•
matrix obtained by removing row i and column j
from A.
Cofactor Cij = (-1)i+jMij
If A is a 1x1 matrix then det(A)=a11.
In general,
n
det( A)   aijCij
j 1
where i can be any value i=1,…n.
A Few Important Properties
• det(AB) = det(A)det(B)
• If T is a triangular matrix,
•
•
•
•
det(T) = t11t22…tnn
det(AT) = det(A)
If A is singular then det(A)=0.
If A is invertible then det(A)=0.
If the pivots from Gaussian Elimination are d1,
d2,…,dn then
det(A) = d1d2=dn
where the plus or minus sign depends on
whether the number of row exchanges is even
or odd.
Characteristic Equation
• Ax = x can be written as
•
(A-  I)x = 0
which holds for x=0, so (A-  I) is singular and
det(A-  I) = 0
This is called the characteristic polynomial. If A
is nxn the polynomial is of degree n and so A
has n eigenvalues, counting multiplicities.
Example
 4 3

A  
 1 2
det( A  I )  0 
3 
4  

A  I  
2
 1
(4   )( 2   )  (1)(3)  0
2  6  5  0  (  5)(  1)  0
• Hence the two eigenvalues are 1 and 5.
Example (continued)
• Once we have the eigenvalues, the eigenvectors
can be obtained by substituting back into
(A-  I)x = 0.
• This gives eigenvectors (1 -1)T and (1 1/3)T
• Note that we can scale the eigenvectors any way
we want.
• Determinant are not used for finding the
eigenvalues of large matrices.
Positive Definite Matrices
• A complex matrix A is positive definite if
for every nonzero complex vector x the
quadratic form xHAx is real and:
xHAx > 0
where xH denotes the conjugate transpose
of x (i.e., change the sign of the imaginary
part of each component of x and then
transpose).
Eigenvalues of Positive
Definite Matrices
• If A is positive definite and  and x are an
eigenvalue/eigenvector pair, then:
Ax =  x  xHAx =  xHx
• Since xHAx and xHx are both real and
positive it follows that  is real and
positive.
Properties of Positive
Definite Matrices
• If A is a positive definite matrix then:
– A is nonsingular.
– The inverse of A is positive definite.
– Gaussian elimination can be performed on A
without pivoting.
– The eigenvalues of A are positive.