Download The solution of a system of equations

Linear Systems of Equations
Ax = b
Marco Lattuada
Swiss Federal Institute of Technology - ETH
Institut für Chemie und Bioingenieurwissenschaften
ETH Hönggerberg/ HCI F135 – Zürich (Switzerland)
E-mail: [email protected]
http://www.morbidelli-group.ethz.ch/education/index
Definition of the Problem
We want to solve:
A ( n , n )  x ( n ,1)  b ( n ,1)
where x is the vector of the unknowns, while A and b are given.
Hypotheses:
• A-1 exists
1. The number of equations is•equal
to the
number of unknowns
A is not
singular
(that is, A is a square matrix)
• A's columns are linearly independent
1. The coefficients of A, b and• xA's
arelines
real are linearly independent
• det(A)
2. The solution of the system exists
andisitnon-zero
is unique
• rank(A) is equal to n
• Ax = 0 only if x is a null vector
Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers
Solution of Linear Systems of Equations – Page # 2
Analytical Approach
Cramer’s rule (1750):
The solution of a system of equations:
A x  b
Is given by:
det  A i 
xi 
det  A 
 a1,1 a1,2

a2,1 a2,2

Ai 


a a
 n ,1 n ,2
where Ai is defined as follows:
a1,i 1 b1 a1,i 1
a2,i 1 b2 a2,i 1
an ,i 1 bn an ,i 1
a1,n 

a2,n 


an ,n 
Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers
Solution of Linear Systems of Equations – Page # 3
b replaces the ith column
Calculation of the determinant
How to compute the determinant of a square matrix?
Laplace formula (1772):
N
det( A )   a1,iC1, j
i 1
where Ci,j is the cofactor of element ai,j. The cofactor Ci,j is the determinant of
the submatrix obtained by removing the ith row and the jth column of the
matrix, multiplied by (-1)i+j: a1,1 a1,2
a1, j 1 a1, j 1 a1,n



i j


Ci , j   1 det Mi , j
th
 ai 1,1 ai 1,2
ai 1, j 1 ai 1, j 1 ai 1,n  No j
 column
Mi , j  
ai 1, j 1 ai 1, j 1 ai 1,n 
 ai 1,1 ai 1,2


th
No i row


a a

a
a
a
n
,1
n
,2
n
,
j

1
n
,
j

1
n
,
n




Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers
Solution of Linear Systems of Equations – Page # 4
Numerical approach: Gauss Elimination Method
Let us consider the system:
3 2 1   x1  1 
1 2 2   x   5 

  2  
1 1 1   x3   2
Let us consider the following operations:
1. I multiply one line by a constant
2. I substitute one line with a linear combination of the others
3. I operate a permutation of the lines
The result does not change
Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers
Solution of Linear Systems of Equations – Page # 5
Gauss Elimination Method
Numerical Example
Triangular System
• Multiply by -3
Multiply
by-4
-3
• Sum •it• Multiply
to
1st lineby
•• Sum
Sumititto
to2nd
1st line
Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers
Solution of Linear Systems of Equations – Page # 6
Gauss Elimination Method
General Case
 a11 a12
A   a21 a22
 a31 a32
a13 
a23 
a33 
I want to replace a21 with a zero
I define the multiplier l21:
l21 
Note that:
a21
a11
 1 0 0  a11 a12
 l
 a
1
0
 21
  21 a22
 0 0 1   a31 a32
Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers
Solution of Linear Systems of Equations – Page # 7
( new )
( old )
a21
 a21
 l21a11  0
 ( new)
( old )
 a22
 a22
 l21a12
a ( new)  a ( old )  l a
23
21 13
 23
a13   a11 a12
( new )
a23    0 a22
a33   a31 a32
a13 
( new ) 
a23

a33 
Gauss Elimination Method
Gauss Transformation Matrix
M
(0)
A
(0)
where:
 1
(0)
  l21
 l31(0)
lij( k ) 
0 0   a11(0)
 (0)

1 0    a21
(0)
0 1   a31
a12(0)
(0)
a22
(0)
a32
a13(0)   a11(0)

(0) 
a23    0
(0) 

a33
  0
a12(0)
(1)
a22
(1)
a32
a13(0) 
(1)
(1) 
a23   A
(1) 
a33

aij( k )
a (jjk )
Solution:
0
0   a11(0) a12(0) a13(0)   a11(0) a12(0) a13(0) 
1

( n 1)
( n  2)   (1)
(0)
(1)
(1)
(2)

(1) (0) 
(1) 
(1)
(1) 
A

M
M
M
A
M  A  0
1
0    0 a22 a23   ( n 01) a22( n 1) a23   A
A
xb
(1) (0) 
(1)
(2) 
(n 0
1) l (1)( n  2)
(1)
(0)




1
0
a
a
0
0
a
21
33 
 M M 32b 33  
b M
Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers
Solution of Linear Systems of Equations – Page # 8
Gauss Elimination Method
Numerical Example
Total number of
operations required
n3
 j ( j  1)   j 
j 2
j 2
3
n 1
n 1
2
(n-1)(n-2)
operations(flops)
(flops)
n(n-1) operations
Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers
Solution of Linear Systems of Equations – Page # 9
Gauss Transformation Method
Gauss Elimination Method
1. Changes the matrix A
2. Needs the coefficient vector b
3. Must re-run the method if b is changed
Let us change our point of view!
MM
( n 2)
(1)
M M
Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers
Solution of Linear Systems of Equations – Page # 10
(0)
can be used to transform A
Gauss Transformation Method
Properties
• The final matrix A is a right triangular matrix
• The matrix M is a left triangular matrix
• The inverse of M is also a left triangular matrix
• The matrix L = M-1 has the simple form:
1
1
(0)
L  M  l21
l31(0)
Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers
Solution of Linear Systems of Equations – Page # 11
0
1
l31(1)
0
0 
1 
LR (LU) Factorization
Consider the following expression:
A
( n 2)
 MA
R  MA
(0)
(0)
Right triangular
Let us multiply by L = M-1 both sides:
1
LR  M MA
(0)
A
(0)
Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers
Solution of Linear Systems of Equations – Page # 12
A  LR
LR Factorization
Starting matrix A is transformed (factorized) as:
A  LR
Let us solve a linear system with a generic vector b:
 Ly  b
Ax  LRx  b  
Rx  y
1. For every vector b, two simple triangular systems must
be solved without factorizing again
2. The matrices LR can be stored using the elements of A
3. If A is modified, it is often possible to modify L and R
accordingly without factorizing
Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers
Solution of Linear Systems of Equations – Page # 13
Problems of Gaussian Elimination and LR
Starting matrix:
A
(0)
 a11(0)
 (0)
  a21
(0)
 a31

a12(0)
(0)
a22
(0)
a32
a13(0) 
(0) 
a23 
(0) 
a33

Pivot value must be ≠ 0
Consider the following system:
 x2  1

0.4 x1  0.3x2  0.1
a11 = 0  I switch the lines
 x1 = 1 and x2 = 1
Consider the following similar system:
2 1020 x1  x2  1

0.4 x1  0.3x2  0.1
Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers
Solution of Linear Systems of Equations – Page # 14
Manual LR Factorization
without pivoting