Download Linear Systems

Numerical Computation
Lecture 5: Linear Systems – part I
United International College
Review
• During our Last Class we covered:
– Root Finding: More on Newton’s Method, Secant
Method
Today
• We will cover:
– Linear Systems: Gaussian Elimination
– Section 3.1 in Pav
– Sections 2.1, 2.2, 2.3 in Moler
Linear Systems
a11 x1  a12 x2  a13 x3    b1
a21x1  a22 x2  a23 x3    b2
a31x1  a32 x2  a33 x3    b3

 a11

a21
a
 31
 
a12
a13
a22
a23
a32
a33


  x1   b1 
   
  x2  b2 
 



 x3
b3
   
      
Linear Systems
• Solve Ax=b, where A is an nn matrix and
b is an n1 column vector
• We can also talk about non-square systems
where
A is mn, b is m1, and x is n1
– Overdetermined if m>n:
“more equations than unknowns”
– Underdetermined if n>m:
“more unknowns than equations”
Singular Systems
• A is singular if some row is
linear combination of other rows
• Singular systems can be underdetermined:
2 x1  3x2  5
or inconsistent:
4 x1  6 x2  10
2 x1  3x2  5
4 x1  6 x2  11
Using Matrix Inverses
• To solve Ax = b it would be nice to use the
inverse to A, that is, A-1
• However, it is usually not a good idea to
compute x=A-1b
– Inefficient
– Prone to roundoff error
Gaussian Elimination
• Fundamental operations:
1.Replace one equation with linear combination
of other equations
2.Interchange two equations
3.Multiply one equation by a scalar
• These are called elementary row
operations.
• Do these operations again and again to
reduce the system to a “trivial” system
Triangular Form
• Two special forms of matrices are especially
nice for solving Ax=b:
• In both cases, successive substitution leads
to a solution
Triangular Form
• A is lower triangular
 a11

a21

a31
a
 41
 
0
0
0

a22
0
0

a32
a33
0

a42
a43
a44





b1 

b2 

b3 
b4 

 
Triangular Form
• Solve by forward substitution:
 a11

a21

a31
a
 41
 
0
0
0

a22
0
0

a32
a33
0

a42
a43
a44





b1
x1 
a11
b1 

b2 

b3 
b4 

 
Triangular Form
• Solve by forward substitution:
 a11

a21

a31
a
 41
 
0
0
0

a22
0
0

a32
a33
0

a42
a43
a44





b2  a21x1
x2 
a22
b1 

b2 

b3 
b4 

 
Triangular Form
• Solve by forward substitution:
 a11

a21

a31
a
 41
 
0
0
0

a22
0
0

a32
a33
0

a42
a43
a44





b3  a31x1  a32 x2
x3 
a33
b1 

b2 

b3 
b4 

 
Etc
Triangular Form
• If A is upper triangular, solve by backsubstitution:
a11

0

0
0

 0
a12
a13
a14
a15
a22
a23
a24
a25
0
a33
a34
a35
0
0
a44
a45
0
0
0
a55
b5
x5 
a55
b1 

b2 

b3 
b4 

b5 
Triangular Form
• Solve by back-substitution:
a11

0

0
0

 0
a12
a13
a14
a15
a22
a23
a24
a25
0
a33
a34
a35
0
0
a44
a45
0
0
0
a55
b4  a45 x5
x4 
a44
b1 

b2 

b3 
b4 

b5 
Etc
Gaussian Elimination Algorithm
• Do elementary row operations on the
augmented system [A|b] to reduce the
system to upper triangular form.
• Then, use back-substitution to find the
answer.
Gaussian Elimination
Gaussian Elimination
• Example:
• Augmented Matrix form:
Gaussian Elimination
• Row Ops: What do we do to zero out first
column under first pivot?
• Zero out below second pivot:
Gaussian Elimination
• Back-substitute
Gaussian Elimination
Gaussian Elimination - Pivoting
• Consider this system:
0

2
1
3
2

8
• We immediately run into a problem:
we cannot zero out below pivot, or backsubstitute!
• More subtle problem: 0.001 1 2

 2
3

8
Gaussian Elimination - Pivoting
• Conclusion: small diagonal elements are
bad!
• Remedy: swap the row with the small
diagonal element with a row below, this is
called pivoting
Gaussian Elimination - Pivoting
0

2
• Our Example:
2

8
1
3
• Swap rows 1 and 2:
2

0
8

2
3
1
• Now continue:
1

0
3
2
1
4

2
1

0
0
1
1

2
Gaussian Elimination - Pivoting
• Two strategies for pivoting:
– Partial Pivoting
– Scaled Partial Pivoting
Partial Pivoting
Scaled Partial Pivoting
Pav, Section 3.2
Practice
• Class Exercise: We will work through
the example in Pav, Section 3.2.2
Practice
• Class Exercise: Do Pav Ex 3.7 for partial
pivoting (“naïve Gaussian
Elimination”).
• Do Pav Ex 3.7 for scaled partial
pivoting.
Matlab Implementation
• Task: Implement Gaussian Elimination
(without pivoting) in a Matlab M-file.
• Notes
• Input = Coefficient matrix A, rhs b
• Output = solution vector x
Matlab Implementation
• Class Exercise: We will go through this
code line by line, using the example in
Pav section 3.3.2 to see how the code
works.
Matlab Implementation
• Matlab:
>> A=[2 1 1 3; 4 4 0 7; 6 5 4 17; 2 -1 0 7]
A=
2 1 1 3
4 4 0 7
6 5 4 17
>> gauss_no_pivot(A,b)
2 -1 0 7
>> b = [7 11 31 15]'
ans =
b=
1.5417
7
-1.4167
11
0.8333
31
1.5000
15
Matlab Implementation
• Class Exercise: How would we change
the Matlab function gauss_no_pivot so
we could see the result of each step of
the row reduction?
Matlab – Partial Pivoting
• Partial Pivoting:
• At step k, we are working on kth row,
pivot = Akk . Search for largest Aik in kth
column below (and including) Akk . Let p
= index of row containing largest entry.
• If p ≠ k, swap rows p and k.
• Continue with Gaussian Elimination.
Matlab – Partial Pivoting
• Finding largest entry in a column:
>> A
A=
2 1 1 3
4 4 0 7
6 5 4 17
2 -1 0 7
>> [r,m] = max(A(2:4,2))
r=
5
m=
2
• Why isn’t m = 3?
Matlab – Partial Pivoting
• Swapping rows m and k:
BB =
2 1 1 3
4 4 0 7
6 5 4 17
2 -1 0 7
>> BB([1 3],:) = BB([3 1], :)
BB =
6 5 4 17
4 4 0 7
2 1 1 3
2 -1 0 7
Matlab – Partial Pivoting
• Code change?
• All that is needed is in main loop (going
from rows k ->n-1) we add
% Find max value (M) and index (m) of max entry below AAkk
[M,m] = max(abs(AA(k:n,k)));
m = m + (k-1); % row offset
% swap rows, if needed
if m ~= k, AA([k m],:) = AA([m k],:);
end
Matlab – Partial Pivoting
• Class Exercise
• Review example in Moler Section 2.6