Download Matrix inversion

Matrix Primer
Reference: http://femur.wpi.edu/Learning-Modules/Linear-Algebra/home/
Definitions
 Matrix: An m  n matrix is a rectangular array of numbers with m rows and n
columns. Only real number entries will be considered in this tutorial. A general
matrix will be represented by an underlined uppercase letter and a row or column
matrix is represented by an underlined lowercase letter.
Example:
1 2 3
A= 4 5 6
1
b = (1 2 3 )
c=
7 8 9

2
3
Matrix element: A matrix element aij is the element in the ith row and jth column.
Example:
1 4 3
5 6 2
3 0 9
6 7 3
a22 = 6,
a32 = 0, a41 = 6
Special Matrices
 Column matrix (column vector) and Row matrix (row vector):
Example: See matrix b and matrix c above.

Square matrix: m = n
Example: See matrix A above.

Diagonal matrix: A square matrix with all zer0 elements except the diagonal terms.
Example

Identity matrix (unity matrix): A square matrix with the diagonal elements equal 1
and remaining elements equal to 0. It is designated by a symbol I.
Example:
1 0 0
I=
0 1 0
0 0 1
1

Inverse of a matrix: B is the inverse of A if A B = B A = I. If B is the inverse of A,
then A is also the inverse of B. Both matrices are invertible.
A A-1 = A-1 A = I
(A-1) -1 = A

Transpose of a matrix: The transpose of a matrix results from the interchanging of the
rows and the columns of the matrix. It is designated by the superscript T or ‘.
Example:
1 2
3
4
5 6
7
8
1 5 9
T
2 6 10
=
3 7 11
9 10 11 12

Symmetric matrix: AT = A (Note: aij = aji)
Example:
1
3 2
T
2
1
=
3 8 5

4 8 12
5 9
3 8 5
2
5 9
Skew symmetric matrix: AT = -A (Note: aij = -aji  The diagonal elements = 0)
Example:
0 3
2
3 0
5
T
=
2 5 0

3 2
0
3 2
3
0 5
2
5 0
Triangular matrix: (must be a square matrix)
Example:
1 2 3
Upper:
1 0 0
Lower:
0 4 5
2 3 0
0 0 6

4 5 6
Zero or null matrix: All matrix elements are zero.
Matrix transposition
 Properties:
o (AT)T = A
o (kA)T = k(AT)
o (A + B)T = AT + BT
o (A B)T = BT AT
and
(A B C)T = CT BT AT
2
Matrix inversion
 Properties:
o (A-1)-1 = A
o (A B )-1 = B-1A-1
o (kA)-1 = (1/k)A-1
o (AT)-1 = (A-1)T

o If A is invertible, then x = A-1 b is the unique solution for A x = b
Methods:
o Diagonal matrix:
1 0 0
x

1
x 0 0


0 y 0 =
1

0
0



y

0 0 z

1
0 0

z 

o Triangular matrix:
1

0
0 

x
1
 1 0


 x 0 0
1


x
0
x
w
1

 =



=
0
w y 0







(
x

y
)
y

y z
 y 1 
v u z


w u  y  v u 1 
 ( x z) z 


( y  z) z 
 ( x y z)
1
x y =


0 z
 1 y 
 x ( x z) 


0 1 
z 

1
x w v
0 y u =


0 0 z
o Row reduction: (A | I)  (I | A-1)
Example:
1 0 0
1 1 1 0
3 0 0 1
1
3
=
0 1 1
1
3
o Determinant:
 
T
adj ( A) cij
A 

| A|
| A|
1
a
For a 2  2 matrix :  11
a21
 a 22  a12 
 a
a12 
a11 
21


a 22 
a11a22  a12 a21
1
3
 1  w w u  y v 
 x ( x y) ( x y z) 


u
0 1


y
( y  z) 


1
0 0

z


Linear dependence
 Definition: If there exists a unique solution (Ki = 0) to the equation set
K1U1 + K2U2 + … + KnUn = 0,

then the vector set (U1, U2, … , Un) is linearly independent.
Example:
o U1 = (1,5) U2 = (3,2)
K1 + 3K2 = 0
5K1 + 2K2 = 0
 K1 = K2 = 0 is the only solution  U1and U2 are linearly independent.
o U1 = (1,5) U2 = (2,10)
K1 + 2K2 = 0
5K1 + 10K2 = 0
 There are infinite number of solutions  U1and U2 are linearly dependent.
Addition and subtraction
 Properties:
o Additions and subtractions are performed on each respective elements of the
matrix.
o The order of the matrices must be the same.
o A + B = B +A
o A + B + C = (A + B) + C = A + (B + C)
o A+0=0+A=A
o A – B = A + (-B)
Multiplication and division
 Scalar multiplication and division: Multiplication or division is performed on each
element of the matrix.

Matrix multiplication: X Y = Z
X–mt
Y–tn
Z–mn
t
zij =  xik ykj
k=1
a11 a12 a13
a21 a22 a23
b11 b12
 b21 b22
b31 b32
=
( a11  b11
( a21  b11
a12  b21
a22  b21
4
a13  b31 ) ( a11  b12
a23  b31 ) ( a21  b12
a12  b22
a22  b22
a13  b32 )
a23  b32 )

Properties:
o
o
o
o
o
ABBA
A (B C) = (A B) C
A (B + C) = A B + A C, (A + B) C = A C + B C
a (B C) = B (a C) = (a B) C
A  B = A B-1
Solving linear equations
 Formulation:
a11 x1 + a12 x2 + a13 x3 = b1
a21 x1 + a22 x2 + a23 x3 = b2
a31 x1 + a32 x2 + a33 x3 = b3
Matrix form: A x = b
where
a11 a12 a13
x1
b1
A = a21 a22 a23
x = x2
b = b2
a31 a32 a33
x3
b3

Solution possibilities:
o Unique solution
o Infinite solutions
o No solution

Solution methods:
o Algebraic (addition/subtraction, substitution, comparison, …)
o Graphical (2-dimensional only)
o Matrix inversion (or Gaussian elimination—lecture example): x = A-1 b
o Cramer’s rule (determinant): xi = |Ai| / |A| ,
where |Ai| is found by substituting the ith column of A with b. If |A| = 0 then it
is called a singular matrix. A singular matrix has no inverse.
a11 a12
a21 a22
= a11  a22 a12  a21
a11 a12 a13
a21 a22 a23
=
a31 a32 a33
a11  a22  a33
a11  a23  a32
a21  a12  a33
5
a21  a13  a32
a31  a12  a23
a31  a13  a22
Determinants
 Definition:
For an n  n matrix using any column j :
n
A   aij cij
i 1
where cij is the cofactor correspond ing to aij :
cij  (1) i  j M
where M is the (n-1 )  (n-1 ) submatrix of A with i th row and jth column deleted.

Diagonal or triangular: |A| equals to the product of all diagonal entries.

Minor: Any square submatrix A is called a minor of A.
o Principal minors:
 a11 a12 a13 
 a11 a12 a13 
 a11 a12
a



a
 21 a22 a23 
a21 a22 a23 
 21 a22
 a31 a32 a33 
a31 a32 a33 
 a31 a32
o Leading principal minors:
 a11 a12
 a11 a12 
a
a11 

 21 a22
a
a
22 
 21
a31 a32

Matrix inversion:
 a11
a
 21

a13 
a23 
a33 
 T
adj ( A) cij
A 

| A|
| A|
For a 2  2 matrix :
1

a13 
a23 
a33 
a12 
a22 
1
 a22  a12 
 a
a11 
  21
a11 a22  a12 a21
Properties of determinant
o Non-invertible if |A| = 0
o |A| = 0 if any row of A is filled entirely with zeros.
o |A| = 0 if any two rows of A are equal or proportional to each other.
o |A| =|AT|
o |A| |B| = |A B|
o |I| = 1
o |A| changes sign when two rows are exchanged.
o |c A| = cn |A| where n is the order of A.
6
Eigenvalue
 Definitions: A x =  x
o Eigenvalue: A value  for which the above equation set has and a non-trivial
solution.
o Eigenvector: A vector of A corresponding to .
o Characteristic determinant/equation: D() = |A –  I | = 0.  can be determined
from this equation.

Example:
5
4
5 4
A
 D ( ) 
 2  7  6  0

1
2
1
2




 1  1, 2  6 (eigenvalu es)
5  1
1  1  
 1
x x 0
4   x11  0
    11 12



x11  x12  0
2  1  x12  0
1
 (One of infinite possibilit ies) x1T   
 1
 x21  4 x22  0
4   x21  0
5  6
2  6  
  



x21  4 x22  0
2  6  x22  0
 1
 4
(Solving either equation)  x21  4 x22  (One of infinite possibilit ies) x 2T   
1 
(Solving either equation)  x11   x12
 1/ 2 
x
x1
 xˆ 1T  1 


x1
12  (1) 2  1 / 2 


 1/ 2
 xˆ   xˆ 1T | xˆ 2T  
 1 / 2
Matrix Calculus:

4 / 17 

1 / 17 
xˆ 2T 
4 / 17 
x2
x2



x2
12  4 2 1 / 17 
(eigenvecto rs - normalized )
Differentiation: Term by term
 t
2t 2

d
sin( t ) 1 / t
dt  2
3t
 t

and
4   1
4t
0 
 
3t   cos(t ) ln( t )
3 
cos(t )  2t
3
 sin( t )

Integration: Term by term
 t
4t
0 
2t 2
 1



 cos(t ) ln( t ) 3  dt  sin(2 t ) 1/ t
 t
 2t
3
 sin( t )
3t

7
4 

3t 
cos(t )

Cayley-Hamilton theorem:
Theory:
For a 3  3 matrix A:
f ( A)  1  I   2  A   3  A
2
 1  1 0

where   2   1 1
  3  1 2

0 2 

12 
2 2 
1
 f (0 )
  f (1 ) 
 f (2 )
Example: Find A10, where
1 2
A
 in terms of f ( A)  1  I   2  A
3 4
Steps:
1. Determine the eigenvalues of the matrix.
2
2. Find the  values.
 0
 00
 1
 1  0 0 01 
01   1   f (0 )





  

    0
11    2   f (1 ) 
1 
 2   1
3. Evaluate f(A).
8
1
 f (0 )


 f (1 ) 
Another example: A100
 0.1 0.3 0.6 
A   0.4 0.5 0.1 
 0.3 0.5 0.2 


 0 
 
  1   eigenvals ( A)
 
 2 
 0  
1

  
  1    0.1  0.1i 
   0.1  0.1i 
 2 
One more example (This one cannot be computed directly): sin(A)
The answer can be obtained by expanding the sin function into a Taylor series:
Since the 3-term series and 4-term series have the same answer, we know the series has
converged.
9
Now use the Cayley-Hamilton theorem:
Conclusion: The Cayley-Hamilton theorem allows us to evaluate matrix functions with a
finite series. Although matrix functions can also be evaluated using the Taylor series
expansion techniques, it requires the evaluation of an infinite series, which often
converges slowly especially for large numbers.
Homework:
Use Matlab to evaluate the following matrix function:
 0.2 0.3 0.5 


f ( A)  cos 0.5 0.1 0.4 
 0.8 0.1 0.1 


a) Use the Taylor series expansion technique. You need to include enough terms to
ensure convergence.
b) Use the Cayley-Hamilton technique.
You need to

Clearly present the results for each step (, , …).

Include all necessary Matlab printouts.

Circle the important steps and results on the printouts.
10

More matrix theory:
Linear dependence (text p. 45):
o Definition: A set of vectors {x1, x2, …, xm} in  n is linearly independent iff
1 x1   2 x 2  ...   m x m  0  1   2  ...   m  0
o Examples:
 x1 = [1 0]T and x2 = [1 2]T
1(1) + 2(1) = 0 and 1(0) + 2(2) = 0  1 = 2 = 0

 they are linearly independent
x1 = [1 1]T and x2 = [2 2]T
1(1) + 2(2) = 0 and 1(1) + 2(2) = 0  1 = -22

 they are linearly dependent
Rank and nullity of a matrix (text p. 48):
o Definition: The rank of A is the number of linearly independent columns in A.
o The nullity of a matrix A is the difference between the total number of
columns and the number of linearly independent columns in the matrix A.
o | A| = 0  rank deficiency
| A|  0  full rank
o Matlab usage: rank(a)
o Example:
1 2 3
rank 4 5 6
1 2 3
2
rank 4 5 5
7 8 9


3
7 8 9
Diagonal form and Jordan form (text p. 55): A square matrix A can be transformed
into a diagonal or block diagonal form:
1
Aˆ  Q A Q where Q  [q1 q 2 ... q n ] and q1 q 2 ... q n are eigenvectors of A.
o Distinct real s: All real diagonal elements, every element corresponds to a .
o Distinct complex s: All real/complex elements, every element corresponds to
a . Additional transform can remove the imaginary parts but the transformed
matrix becomes block diagonal (modal form).
o Repeated s (text p. 60): If the s are not all distinct, the resulting matrix may
comprise upper triangular blocks along the diagonal (Jordan form).
o Matlab usages (see p. 61): eig(a) and Jordan(a)
Norm of a vector (P. 46): The generalized length (magnitude) of the vector
n
o Definition: x k  k  xi k
i 1
o Examples:
n
l1 norm: x 1   xi , l2 norm: x 2 
i 1
n
 xi
i 1
2
, l norm: x   max i xi
o Properties:
 ||x||  0
 ||ax|| = |a|  ||x|| for real a.
 ||x1 + x2||  ||x1|| + ||x2|| (triangular inequality)
11



o Matlab usages: norm(x, 1), norm(x, 2) = norm(x), and norm(x, inf)
Norm of a matrix (P. 78): magnification capability Amn
o ||A||1 = Largest column absolute sum
o ||A||2 = Largest singular value
o ||A|| = Largest row absolute sum
o Matlab usages: norm(a, 1), norm(a, 2) = norm(a), and norm(a, inf)
Singular-value decomposition (SVD):
o Eigenvalues/vectors are defined for square matrices only.
o Non-square matrices are important in linear system analysis
(controllability/observability)
o For an mn matrix H, we can define a symmetric matrix (nn) M = HT H and
the eigenvalues of M are real and nonnegative (positive semidefinite).
o The eigenvalues of M are called the singular values of H (example 3.13, p.76).
o SVD: H can be decomposed into a product of 3 matrices (example 3.14, p.77).
 H = R S QT
 R RT = RT R = Im Q QT = QT Q = In
 S is an mn matrix with the singular values of H on the diagonal.
o Matlab usages:
 s = svd(H) gives the singular values of H.
 [R, S, Q] = svd(H) gives all 3 matrices of the decomposition.
o Applications of SVD:
 Norm of a matrix: ||A||2 = 1 (Largest singular value)
 Rank of a matrix: equal to the number of non-zero singular values.
 Condition number = max / min: Indicates how close a matrix is to rank
deficiency (How much numerical error is likely to be introduced by
computations involving the matrix). Large condition numbers imply
ill-conditioned systems and small (close to 1) condition numbers imply
well-conditioned systems.
 Matlab usage: cond(a)
Lyapunov theorem
o The Lyapunov theorem provides an alternate means to check the asymptotic
stability of a system.
o Lyapunov equation (text p. 70-1):
A(M) = A M + M B = C
o A(M) =  M, where ’s are the eigenvalues of A and they represent all
possible sums of the eigenvalues of A and B.
o A symmetric matrix M is said to be positive definite (denoted by M > 0) if
xTM x > 0 for nonzero x.
o If M > 0, then xTM x = 0 iff x = 0.
o M > 0 iff any one of the following conditions holds:
 Every eigenvalue of M is positive.
 All leading principal minors of M are positive.
 There exists an nn nonsingular matrix such that M = NTN.
o Matlab usage: m = lyap(a, b, -c)
12