Download PPT - Introduction to Matrix Algebra

Autar Kaw
Benjamin Rigsby
http://nm.MathForCollege.com
Transforming Numerical Methods Education for STEM Undergraduates
http://nm.MathForCollege.com
1. solve a set of equations using the Gauss-Seidel
method,
2. recognize the advantages and pitfalls of the GaussSeidel method, and
3. determine under what conditions the Gauss-Seidel
method always converges.
An iterative method.
Basic Procedure:




Algebraically solve each linear equation for xi
Assume an initial guess solution array
Solve for each xi and repeat
Use absolute relative approximate error after each
iteration to check if error is within a pre-specified
tolerance.
4
Why?
The Gauss-Seidel Method allows the user to control round-off error.
Elimination methods such as Gaussian Elimination and LU Decomposition
are prone to prone to round-off error.
Also: If the physics of the problem are understood, a close initial guess can
be made, decreasing the number of iterations needed.
5
Algorithm
A set of n equations and n unknowns:
a11x1  a12 x2  a13 x3  ...  a1n xn  b1
a21x1  a22 x2  a23 x3  ...  a2n xn  b2
.
.
.
.
.
.
an1x1  an 2 x2  an3 x3  ...  ann xn  bn
If: the diagonal elements are
non-zero
Rewrite each equation solving
for the corresponding unknown
ex:
First equation, solve for x1
Second equation, solve for x2
6
Algorithm
Rewriting each equation
x1 
c1  a12 x2  a13 x3   a1n xn
a11
From Equation 1
c2  a21 x1  a23 x3   a2 n xn
a22
From equation 2
x2 

xn 1 
xn 


cn 1  an 1,1 x1  an 1, 2 x2   an 1,n  2 xn  2  an 1,n xn
From equation n-1
an 1,n 1
cn  an1 x1  an 2 x2    an ,n 1 xn 1
From equation n
ann
7
Algorithm
General Form of each equation
n
c1   a1 j x j
x1 
j 1
j 1
cn 1 
xn 1 
a11
n
a
j 1
j  n 1
n 1, j
xj
an 1,n 1
n
c n   a nj x j
n
c2   a2 j x j
x2 
j 1
j 2
a 22
xn 
j 1
j n
a nn
8
Algorithm
General Form for any row ‘i’
n
ci   aij x j
xi 
j 1
j i
aii
, i  1,2,, n.
How or where can this equation be used?
9
Solve for the unknowns
Assume an initial guess for [X]
 x1 
x 
 2
  
 
 xn-1 
 xn 
Use rewritten equations to solve for
each value of xi.
Important: Remember to use the
most recent value of xi. Which
means to apply values calculated to
the calculations remaining in the
current iteration.
10
Calculate the Absolute Relative Approximate Error
a i
xinew  xiold

100
new
xi
So when has the answer been found?
The iterations are stopped when the absolute relative approximate
error is less than a prespecified tolerance for all unknowns.
11
The upward velocity of a rocket is
given at three different times
Table 1 Velocity vs. Time data.
Time, t s Velocity v m/s 
5
106.8
8
177.2
12
279.2
The velocity data is approximated by a polynomial as:
vt   a1t 2  a2t  a3 , 5  t  12.
12
Using a Matrix template of the form
The system of equations becomes
Initial Guess: Assume an initial guess of
t12
2
t 2
t32

t1 1  a1   v1 
   
t 2 1 a2   v2 
t3 1  a3  v3 
 25 5 1  a1  106.8 
 64 8 1 a   177.2 

  2 

144 12 1 a3  279.2
 a1  1
 a    2
 2  
 a3  5
13
Rewriting each equation
106.8  5a 2  a3
a1 
25
 25 5 1  a1  106.8 
 64 8 1 a   177.2 

  2 

144 12 1 a3  279.2
177.2  64a1  a3
a2 
8
279.2  144a1  12a 2
a3 
1
14
Applying the initial guess and solving for ai
 a1  1
 a    2
 2  
 a3  5
Initial Guess
106.8  5(2)  (5)
a1 
 3.6720
25
177.2  643.6720  5
a2 
 7.8510
8
279.2  1443.6720  12 7.8510
a3 
 155.36
1
When solving for a2, how many of the initial guess values were used?
15
Finding the absolute relative approximate error
a i
xinew  xiold

100
new
xi
a 1 
a
2
3.6720  1.0000
x100  72.76%
3.6720
 7.8510  2.0000

x100  125.47%
 7.8510
a 3 
At the end of the first iteration
 a1   3.6720 
a    7.8510
 2 

 a3    155.36 
The maximum absolute relative
approximate error is 125.47%
 155.36  5.0000
x100  103.22%
 155.36
16
Using
 a1   3.6720 
a    7.8510
 2 

 a3    155.36 
Iteration #2
the values of ai are found:
106.8  5 7.8510  155.36
a1 
 12.056
25
from iteration #1
a2 
177.2  6412.056  155.36
 54.882
8
a3 
279.2  14412.056  12 54.882
 798.34
1
17
Finding the absolute relative approximate error
a 1 
a 2 
a 3 
12.056  3.6720
x100  69.543%
12.056
 54.882   7.8510
x100  85.695%
 54.882
 798.34   155.36
x100  80.540%
 798.34
At the end of the second iteration
 a1   12.056 
a    54.882
 2 

a3   798.54
The maximum absolute relative
approximate error is 85.695%
18
Repeating more iterations, the following values are obtained
Iteration
1
2
3
4
5
6
a1
3.6720
12.056
47.182
193.33
800.53
3322.6
a 1 %
72.767
69.543
74.447
75.595
75.850
75.906
a2
a 2 %
a3
−7.8510
−54.882
−255.51
−1093.4
−4577.2
−19049
125.47
85.695
78.521
76.632
76.112
75.972
−155.36
−798.34
−3448.9
−14440
−60072
−249580
a 3 %
103.22
80.540
76.852
76.116
75.963
75.931
Notice – The relative errors are not decreasing at any significant rate
Also, the solution is not converging to the true solution of
19
What went wrong?
Even though done correctly, the answer is not converging to the
correct answer
This example illustrates a pitfall of the Gauss-Siedel method: not all
systems of equations will converge.
Is there a fix?
One class of system of equations always converges: One with a diagonally
dominant coefficient matrix.
Diagonally dominant: [A] in [A] [X] = [C] is diagonally dominant if:
n
n
aii   aij
j 1
j i
for all ‘i’
and
aii   aij
for at least one ‘i’
j 1
j i
20
Diagonally dominant: The coefficient on the diagonal must be at least equal
to the sum of the other coefficients in that row and at least one row with a
diagonal coefficient greater than the sum of the other coefficients in that row.
Which coefficient matrix is diagonally dominant?
 2 5.81 34
A   45 43 1 
123 16
1 
124 34 56 
[B]   23 53 5 
 96 34 129
Most physical systems do result in simultaneous linear equations that have
diagonally dominant coefficient matrices.
21
Given the system of equations
The coefficient matrix is:
12 x1  3x2- 5x3  1
12 3  5
A   1 5 3 
 3 7 13 
x1  5 x2  3x3  28
3x1  7 x2  13x3  76
With an initial guess of
 x1  1
 x   0 
 2  
 x3  1
Will the solution converge using the
Gauss-Siedel method?
22
Checking if the coefficient matrix is diagonally dominant
12 3  5
A   1 5 3 
 3 7 13 
a11  12  12  a12  a13  3   5  8
a22  5  5  a21  a23  1  3  4
a33  13  13  a31  a32  3  7  10
The inequalities are all true and at least one row is strictly greater than:
Therefore: The solution should converge using the Gauss-Siedel Method
23
Rewriting each equation
With an initial guess of
12 3  5  a1   1 
 1 5 3  a   28

  2  
 3 7 13  a3  76
1  3 x 2  5 x3
x1 
12
28  x1  3 x3
x2 
5
76  3 x1  7 x 2
x3 
13
 x1  1
 x   0 
 2  
 x3  1
x1 
1  30  51
 0.50000
12
28  0.5  31
x2 
 4.9000
5
76  30.50000  74.9000
x3 
 3.0923
13
24
The absolute relative approximate error
0.50000  1.0000
a 1 
100  100.00%
0.50000
a
a
2
3
4.9000  0

100  100.00%
4.9000
3.0923  1.0000

100  67.662%
3.0923
The maximum absolute relative error after the first iteration is 100%
25
After Iteration #1
 x1  0.5000
 x   4.9000
 2 

 x3  3.0923
Substituting the x values into the
equations
x1 
1  34.9000  53.0923
 0.14679
12
x2 
28  0.14679  33.0923
 3.7153
5
x3 
76  30.14679   74.900
 3.8118
13
After Iteration #2
26
Iteration #2 absolute relative approximate error
a 1 
a
a
2
3
0.14679  0.50000
100  240.61%
0.14679
3.7153  4.9000

100  31.889%
3.7153
3.8118  3.0923

100  18.874%
3.8118
The maximum absolute relative error after the first iteration is 240.61%
This is much larger than the maximum absolute relative error obtained in
iteration #1. Is this a problem?
27
Repeating more iterations, the following values are obtained
Iteration
a1
a 1 %
a2
a 2 %
a3
1
2
3
4
5
6
0.50000
0.14679
0.74275
0.94675
0.99177
0.99919
100.00
240.61
80.236
21.546
4.5391
0.74307
4.9000
3.7153
3.1644
3.0281
3.0034
3.0001
100.00
31.889
17.408
4.4996
0.82499
0.10856
3.0923
3.8118
3.9708
3.9971
4.0001
4.0001
a 3 %
67.662
18.876
4.0042
0.65772
0.074383
0.00101
 x1  0.99919
The solution obtained  x2    3.0001  is close to the exact solution of
 x3   4.0001 
 x1  1
 x   3.
 2  
 x3  4
28
Given the system of equations
3x1  7 x2  13x3  76
x1  5x2  3x3  28
12 x1  3x2  5 x3  1
With an initial guess of
 x1  1
 x   0 
 2  
 x3  1
Rewriting the equations
76  7 x2  13 x3
x1 
3
28  x1  3 x3
x2 
5
1  12 x1  3 x 2
x3 
5
29
Conducting six iterations, the following values are obtained
Iteration
a1
1
2
3
4
5
6
21.000
−196.15
−1995.0
−20149
2.0364×105
−2.0579×105
a 1 %
A2
95.238
0.80000
110.71
14.421
109.83
−116.02
109.90
1204.6
109.89
−12140
109.89 1.2272×105
a 2 %
a3
a 3 %
100.00
94.453
112.43
109.63
109.92
109.89
50.680
−462.30
4718.1
−47636
4.8144×105
−4.8653×106
98.027
110.96
109.80
109.90
109.89
109.89
The values are not converging.
Does this mean that the Gauss-Seidel method cannot be used?
30
The Gauss-Seidel Method can still be used
The coefficient matrix is not
diagonally dominant
But this is the same set of
equations used in example #2,
which did converge.
3
A   1
12
12
A   1
 3
7 13 
5 3 
3  5
3  5
5 3 
7 13 
If a system of linear equations is not diagonally dominant, check to see if
rearranging the equations can form a diagonally dominant matrix.
31
Not every system of equations can be rearranged to have a diagonally
dominant coefficient matrix.
Observe the set of equations
x1  x2  x3  3
2 x1  3x2  4 x3  9
x1  7 x2  x3  9
Which equation(s) prevents this set of equation from having a
diagonally dominant coefficient matrix?
32
Summary

Advantages of the Gauss-Seidel Method

Algorithm for the Gauss-Seidel Method

Pitfalls of the Gauss-Seidel Method
33
Questions?
34
For all resources on this topic such as digital audiovisual lectures,
primers, textbook chapters, multiple-choice tests, worksheets in
MATLAB, MATHEMATICA, MathCad and MAPLE, blogs,
related physical problems, please visit
http://numericalmethods.eng.usf.edu/topics/gauss_seidel.html
THE END
http://numericalmethods.eng.usf.edu