Download GAUSSIAN ELIMINATION

GAUSSIAN ELIMINATION
Data Structures II:
Algorithm Design & Analysis
Presented by:
Max Le
Justin Lavorgna
Gaussian
Elimination Itinerary
 Introduction:
Who was Carl Friedrich Gauss ?
 What is Gaussian Elimination ?
 Some Basic Terminology ?

 Main
Presentation:
Backward Substitution method
 LU method
 Compute Determinant of a matrix

 Conclusion:
Which Algorithm is more efficient ?
 Which Algorithm is more practical ?

CARL FRIEDRICH
GAUSS
Who Was He?
Born:
April 30th, 1777 Brunswick
Died: February 23rd,1822 Göttingen
One of the all time great German
Mathematicians. His field of study
consisted of most every aspect in
mathematics today.
Gauss’s work contributed to a variety of
different aspects such as:







Gaussian Elimination (Linear Algebra)
Gaussian Primes & Gauss Sums (Number
Theory)
Gaussian Distribution (Statistics)
Gauss (Electromagnetism)
Gaussian Curvature & Gauss-Bonnet Formula
(Differential Geometry)
Gaussian Quadrature (Numerical Analysis)
Gauss’s Identity (Hypergeometric Functions)
GAUSSIAN
ELIMINATION
What is it?
For complex systems of equations where:
 The number of equations are equal to n
 The number of unknowns are equal to n
We must solve by a process of elimination.
Elimination is implied by reducing
the amount of unknowns and
equations in the system.
The elimination process:
1. Subtract multiples of the first
equation from all other equations
2. The goal is to eliminate the first
variables in each equation.
a11 a12 a13
A=
a21 a22 a23
a31 a32 a33
u11 u12 u13
U=
0 u22 u23
0 0 u33
TERMINOLOGY
Equation: ax + by + cz + dw = h
where:
while:
a,b,c,d, and h are known numbers
x,y,z, and w are unknown numbers
The above conditions are what we call a linear
equation. If h = 0 then the linear equation is
said to be homogeneous. All linear equations
make up a linear system while all homogenous
linear equations make up a homogenous
linear system.
HOW IT WORKS
Given System:
x+y+z
= 0
Equation 1
x - 2y + 2z = 4
Equation 2
x + 2y - z = 2
Equation 3
Make system into a matrix:
x+y+z | 0
x - 2y + 2z | 4
x + 2y - z | 2
Equation 1
Equation 2
Equation 3
Objective:
x+y+z | 0
Equation 1
x - 2y + 2z | 4
Equation 2
x + 2y - z | 2
Equation 3
Kill ‘ X ’ Variable in Equation 2
3. Subtract
fromfrom
all other
rows
Step 1.
Observefirst
firstrow
column
left of
the until a
zero
appears below the leading column.
matrix.
Step 2. If leading term is any number other than 1,
multiply the row by its reciprocal to obtain a 1.
Matrix
Multiple: 1
1x + 1y - 1z | 0
-
x+y+z | 0
1x +1y + 1z | 0
Equation 1
1x - 2y + 2z | 4
1x
Equation 2
1x + 2y - 1z | 2
Equation 3
1
- 3y + z | 4
New Equation 2
__________
x - 2y + 2z | 4
x + 2y - z | 2
‘X’ unknown is DEAD…
Objective:
x+y+z | 0
Equation 1
x - 2y + 2z | 4
Equation 2
x + 2y - z | 2
Equation 3
Kill ‘ X ’ Variable in Equation 3
6. Subtract
fromfrom
all other
rows
Step 4.
Observefirst
firstrow
column
left of
the until a
zero
appears below the leading column.
matrix.
Step 5. If leading term is any number other than 1,
multiply the row by its reciprocal to obtain a 1.
Matrix
Multiple: 1
1x + 1y - 1z | 0
x+y+z | 0
x - 2y + 2z | 4
x + 2y - z | 2
-
1x +1y + 1z | 0
Equation 1
- 3y + 1z | 4
1x + 2y - 1z | 2
Equation 2
1x
__________
Equation 3
Equation 1
y - 2z | 2 New Equation 3
‘X’ unknown is DEAD…
Objective:
x+y+z | 0
Equation 1
x - 2y + 2z | 4
Equation 2
x + 2y - z | 2
Equation 3
Kill ‘ Y ’ Variable in Equation 3
9. Add
second
to thefrom
thirdleft
rowofuntil
Step 7.
Observe
firstrow
column
the a
zero
appears below the leading column.
matrix.
Step 8. If leading term is any number other than 1,
multiply the row by its reciprocal to obtain a 1.
Matrix
Multiple: 3
3y - 6z | 6
x+y+z | 0
1x +1y + 1z | 0
Equation 1
- 3y + 1z | 4
Equation 2
1y
2z
|
2
1y
-__________
- 5z | 10
Equation 3
New Equation 3
x - 2y + 2z | 4
x + 2y - z | 2
‘Y’ unknown is DEAD…
Objective:
Solve for Unknowns using
Backward Substitution
BACKWARD
SUBSTITUTION
Linear System Solution:
Final matrix:
U=
Equation 1
x+y+z | 0
Equation 2
- 3y + z | 4
- 5z | 10 Equation 3
Solving for z in equation 3:
- ___
5 z = ___
-2
10
-5
-5
Solving for y in equation 2:
- ___
3 y =+ -2
(-2) =______
( 4 + 2)
-3
-3
Solving for x in equation 1:
x += (-2)
4 + (-2) =_________
( 0 + 2 + 2)
___
1
1
Objective:
Solve using LU Decomposition
LU DECOMPOSITION
LU Decomposition
Information on LU Method
 Example

LU Decomposition

Given a system A*x = b

We want to find Lp and U matrices such
that Lp*U=A

Solving the system Lp*y=b to solve for y.

Then solving the system U*x=y to get x
OBJECTIVE:
 Given
a system A*x = b
LU Decomposition
Consider the system:
1
1
1
2x + 2y + 1z =
2x + 3y - 2z =
4x + 1y - 2z =
Equation in matrix form for A:
2 2 1
2 3 -2
4 1 -2
x
y
z
=
1
1
1
A
X
=
b
Identity Matrix

Identity Matrix is a matrix with all 1 in its
diagonal and zeros elsewhere
I=
1 0 0
0 1 0
0 0 1
Find the U matrix and keep track of Elementary
matrices
First Operation:
2 2 1
2 3 -2
4 1 -2
Multiply row 1 by -1
Add row 1 to row 2
2 2 1
0 1 -3
4 1 -2
Elementary operation equivalent:
1 0 0
0 1 0
0 0 1
Multiply row 1 by -1
Add row 1 to row 2
1 0 0
-1 1 0
0 0 1
= E1
Find the U matrix and keep track of Elementary
matrices
Second Operation:
2 2 1
0 1 -3
4 1 -2
Multiply row 1 by -2
Add row 1 to row 3
2 2 1
0 1 -3
0 -3 -4
Elementary operation equivalent:
1 0 0
0 1 0
0 0 1
Multiply row 1 by -2
Add row 1 to row 3
1 0 0
0 1 0
-2 0 1
= E2
Find the U matrix and keep track of Elementary
matrices
Third Operation:
2 2 1
0 1 -3
0 -3 -4
Multiply row 2 by 3
Add row 2 to row 3
2 2 1
0 1 -3
0 0 -13
=U
Elementary operation equivalent:
1 0 0
0 1 0
0 0 1
Multiply row 2 by 3
Add row 2 to row 3
1 0 0
0 1 0
0 3 1
= E3
Forming the Lp Matrix
1 0 0
1 0 0
E1 = -1 1 0 E2 = 0 1 0 E2 =
0 0 1
-2 0 1
1 0 0
0 1 0
0 3 1
S = E1*E2*E3
1 0 0
S = -1 1 0
0 0 1
1 0 0
0 1 0
-2 0 1
1 0 0
0 1 0
0 3 1
Forming the Lp Matrix
Product of the elementary matrices
1 0 0
S = -1 1 0
-5 3 1
Inverse of a S will give Lp
1 0 0
1/S = 1 1 0
2 -3 1
= Lp
OBJECTIVE:
 We
want to find Lp and U
matrices such that Lp*U=A
LU Decomposition
We can verify that Lp*U = A
1 0 0
-1 1 0
2 -3 1
Lp
2 2 1
0 1 -3 =
0 0 -13
U
2 2 1
2 3 -2
4 1 -2
=A
LU Decomposition
Recall that Upper Triangular Matrix:
U=
2 2 1
0 1 -3
0 0 -13
Recall that Lower Triangular Matrix:
Lp =
1 0 0
1 1 0
2 -3 1
OBJECTIVE:
 Solving
the system Lp*y=b to
solve for y.
Solve Lp*y = b
The system:
1 0 0
1 1 0
2 -3 1
Lp
y1
y2 =
y3
Y
=
1
1
1
b
Solve for y1, y2 and y3
y1 = 1
y1+y2 = 1
2*y1-3*y2 + y3 = 1
=> y2 = 0
=> y3 = -1
OBJECTIVE:
 Then
solving the system U*x=y
to get x
Solve U*x = y
The system :
2 2 1
0 1 -3
0 0 -13
U
X1
X2 =
X3
X
=
1
0
-1
Y
Solve for x1, x2 and x3
-13*X3 = -1
X2 – 3*X3 = 1
2*X1+2*X2 + X3 = 1
=> X3 = 1/13
=> X2 = 3/13
=> X1 = 3/13
Objective:
Solve for linear system
using Determinant of Matrix
DETERMINANT of
MATRIX
Given Equations:
x+y+z = 0
x - 2y + 2z = 4
x + 2y - z = 2
Equation 1
Equation 2
Equation 3
Make equations into a matrix:
1+1+1
1-2+2
1+2-1
How Determinant works:
n
Formula: det A = Σ sja1j det Aj
j=1
a11 a12
det a21 a22
=
a11 det [ a22 ] – a12 det [ a21 ] = a11-a22 – a12a21
a11 det [ a22 ] – a12 det [ a21 ] =
a11-a22 – a12a21
Matrix:
1 1 1
1 -2 2
1 2 -1
=
det
-
det
+
det
-2 2
1 2
1 -2
1 det
- 1 det
+ 1 det
2 -1
1 -1
1 2
det = 1 ( 2 – 4 ) – 1 ( -1 – 2 ) + 1 (2 – (-2) )
det = -2 + 3 + 4
det = 5
CONCLUSION
Time Complexity

Backward substitution O(n^3)
 LU method O(n^3)
Space Complexity

Backward substitution O(n^2)
 LU method O(n^2)
Application

Gaussian Elimination plays a major role
in solving Linear systems, including:
- Digital Signal processing
- Linear system analysis
- Image processing.
- Etc…