Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
GAUSSIAN ELIMINATION Data Structures II: Algorithm Design & Analysis Presented by: Max Le Justin Lavorgna Gaussian Elimination Itinerary Introduction: Who was Carl Friedrich Gauss ? What is Gaussian Elimination ? Some Basic Terminology ? Main Presentation: Backward Substitution method LU method Compute Determinant of a matrix Conclusion: Which Algorithm is more efficient ? Which Algorithm is more practical ? CARL FRIEDRICH GAUSS Who Was He? Born: April 30th, 1777 Brunswick Died: February 23rd,1822 Göttingen One of the all time great German Mathematicians. His field of study consisted of most every aspect in mathematics today. Gauss’s work contributed to a variety of different aspects such as: Gaussian Elimination (Linear Algebra) Gaussian Primes & Gauss Sums (Number Theory) Gaussian Distribution (Statistics) Gauss (Electromagnetism) Gaussian Curvature & Gauss-Bonnet Formula (Differential Geometry) Gaussian Quadrature (Numerical Analysis) Gauss’s Identity (Hypergeometric Functions) GAUSSIAN ELIMINATION What is it? For complex systems of equations where: The number of equations are equal to n The number of unknowns are equal to n We must solve by a process of elimination. Elimination is implied by reducing the amount of unknowns and equations in the system. The elimination process: 1. Subtract multiples of the first equation from all other equations 2. The goal is to eliminate the first variables in each equation. a11 a12 a13 A= a21 a22 a23 a31 a32 a33 u11 u12 u13 U= 0 u22 u23 0 0 u33 TERMINOLOGY Equation: ax + by + cz + dw = h where: while: a,b,c,d, and h are known numbers x,y,z, and w are unknown numbers The above conditions are what we call a linear equation. If h = 0 then the linear equation is said to be homogeneous. All linear equations make up a linear system while all homogenous linear equations make up a homogenous linear system. HOW IT WORKS Given System: x+y+z = 0 Equation 1 x - 2y + 2z = 4 Equation 2 x + 2y - z = 2 Equation 3 Make system into a matrix: x+y+z | 0 x - 2y + 2z | 4 x + 2y - z | 2 Equation 1 Equation 2 Equation 3 Objective: x+y+z | 0 Equation 1 x - 2y + 2z | 4 Equation 2 x + 2y - z | 2 Equation 3 Kill ‘ X ’ Variable in Equation 2 3. Subtract fromfrom all other rows Step 1. Observefirst firstrow column left of the until a zero appears below the leading column. matrix. Step 2. If leading term is any number other than 1, multiply the row by its reciprocal to obtain a 1. Matrix Multiple: 1 1x + 1y - 1z | 0 - x+y+z | 0 1x +1y + 1z | 0 Equation 1 1x - 2y + 2z | 4 1x Equation 2 1x + 2y - 1z | 2 Equation 3 1 - 3y + z | 4 New Equation 2 __________ x - 2y + 2z | 4 x + 2y - z | 2 ‘X’ unknown is DEAD… Objective: x+y+z | 0 Equation 1 x - 2y + 2z | 4 Equation 2 x + 2y - z | 2 Equation 3 Kill ‘ X ’ Variable in Equation 3 6. Subtract fromfrom all other rows Step 4. Observefirst firstrow column left of the until a zero appears below the leading column. matrix. Step 5. If leading term is any number other than 1, multiply the row by its reciprocal to obtain a 1. Matrix Multiple: 1 1x + 1y - 1z | 0 x+y+z | 0 x - 2y + 2z | 4 x + 2y - z | 2 - 1x +1y + 1z | 0 Equation 1 - 3y + 1z | 4 1x + 2y - 1z | 2 Equation 2 1x __________ Equation 3 Equation 1 y - 2z | 2 New Equation 3 ‘X’ unknown is DEAD… Objective: x+y+z | 0 Equation 1 x - 2y + 2z | 4 Equation 2 x + 2y - z | 2 Equation 3 Kill ‘ Y ’ Variable in Equation 3 9. Add second to thefrom thirdleft rowofuntil Step 7. Observe firstrow column the a zero appears below the leading column. matrix. Step 8. If leading term is any number other than 1, multiply the row by its reciprocal to obtain a 1. Matrix Multiple: 3 3y - 6z | 6 x+y+z | 0 1x +1y + 1z | 0 Equation 1 - 3y + 1z | 4 Equation 2 1y 2z | 2 1y -__________ - 5z | 10 Equation 3 New Equation 3 x - 2y + 2z | 4 x + 2y - z | 2 ‘Y’ unknown is DEAD… Objective: Solve for Unknowns using Backward Substitution BACKWARD SUBSTITUTION Linear System Solution: Final matrix: U= Equation 1 x+y+z | 0 Equation 2 - 3y + z | 4 - 5z | 10 Equation 3 Solving for z in equation 3: - ___ 5 z = ___ -2 10 -5 -5 Solving for y in equation 2: - ___ 3 y =+ -2 (-2) =______ ( 4 + 2) -3 -3 Solving for x in equation 1: x += (-2) 4 + (-2) =_________ ( 0 + 2 + 2) ___ 1 1 Objective: Solve using LU Decomposition LU DECOMPOSITION LU Decomposition Information on LU Method Example LU Decomposition Given a system A*x = b We want to find Lp and U matrices such that Lp*U=A Solving the system Lp*y=b to solve for y. Then solving the system U*x=y to get x OBJECTIVE: Given a system A*x = b LU Decomposition Consider the system: 1 1 1 2x + 2y + 1z = 2x + 3y - 2z = 4x + 1y - 2z = Equation in matrix form for A: 2 2 1 2 3 -2 4 1 -2 x y z = 1 1 1 A X = b Identity Matrix Identity Matrix is a matrix with all 1 in its diagonal and zeros elsewhere I= 1 0 0 0 1 0 0 0 1 Find the U matrix and keep track of Elementary matrices First Operation: 2 2 1 2 3 -2 4 1 -2 Multiply row 1 by -1 Add row 1 to row 2 2 2 1 0 1 -3 4 1 -2 Elementary operation equivalent: 1 0 0 0 1 0 0 0 1 Multiply row 1 by -1 Add row 1 to row 2 1 0 0 -1 1 0 0 0 1 = E1 Find the U matrix and keep track of Elementary matrices Second Operation: 2 2 1 0 1 -3 4 1 -2 Multiply row 1 by -2 Add row 1 to row 3 2 2 1 0 1 -3 0 -3 -4 Elementary operation equivalent: 1 0 0 0 1 0 0 0 1 Multiply row 1 by -2 Add row 1 to row 3 1 0 0 0 1 0 -2 0 1 = E2 Find the U matrix and keep track of Elementary matrices Third Operation: 2 2 1 0 1 -3 0 -3 -4 Multiply row 2 by 3 Add row 2 to row 3 2 2 1 0 1 -3 0 0 -13 =U Elementary operation equivalent: 1 0 0 0 1 0 0 0 1 Multiply row 2 by 3 Add row 2 to row 3 1 0 0 0 1 0 0 3 1 = E3 Forming the Lp Matrix 1 0 0 1 0 0 E1 = -1 1 0 E2 = 0 1 0 E2 = 0 0 1 -2 0 1 1 0 0 0 1 0 0 3 1 S = E1*E2*E3 1 0 0 S = -1 1 0 0 0 1 1 0 0 0 1 0 -2 0 1 1 0 0 0 1 0 0 3 1 Forming the Lp Matrix Product of the elementary matrices 1 0 0 S = -1 1 0 -5 3 1 Inverse of a S will give Lp 1 0 0 1/S = 1 1 0 2 -3 1 = Lp OBJECTIVE: We want to find Lp and U matrices such that Lp*U=A LU Decomposition We can verify that Lp*U = A 1 0 0 -1 1 0 2 -3 1 Lp 2 2 1 0 1 -3 = 0 0 -13 U 2 2 1 2 3 -2 4 1 -2 =A LU Decomposition Recall that Upper Triangular Matrix: U= 2 2 1 0 1 -3 0 0 -13 Recall that Lower Triangular Matrix: Lp = 1 0 0 1 1 0 2 -3 1 OBJECTIVE: Solving the system Lp*y=b to solve for y. Solve Lp*y = b The system: 1 0 0 1 1 0 2 -3 1 Lp y1 y2 = y3 Y = 1 1 1 b Solve for y1, y2 and y3 y1 = 1 y1+y2 = 1 2*y1-3*y2 + y3 = 1 => y2 = 0 => y3 = -1 OBJECTIVE: Then solving the system U*x=y to get x Solve U*x = y The system : 2 2 1 0 1 -3 0 0 -13 U X1 X2 = X3 X = 1 0 -1 Y Solve for x1, x2 and x3 -13*X3 = -1 X2 – 3*X3 = 1 2*X1+2*X2 + X3 = 1 => X3 = 1/13 => X2 = 3/13 => X1 = 3/13 Objective: Solve for linear system using Determinant of Matrix DETERMINANT of MATRIX Given Equations: x+y+z = 0 x - 2y + 2z = 4 x + 2y - z = 2 Equation 1 Equation 2 Equation 3 Make equations into a matrix: 1+1+1 1-2+2 1+2-1 How Determinant works: n Formula: det A = Σ sja1j det Aj j=1 a11 a12 det a21 a22 = a11 det [ a22 ] – a12 det [ a21 ] = a11-a22 – a12a21 a11 det [ a22 ] – a12 det [ a21 ] = a11-a22 – a12a21 Matrix: 1 1 1 1 -2 2 1 2 -1 = det - det + det -2 2 1 2 1 -2 1 det - 1 det + 1 det 2 -1 1 -1 1 2 det = 1 ( 2 – 4 ) – 1 ( -1 – 2 ) + 1 (2 – (-2) ) det = -2 + 3 + 4 det = 5 CONCLUSION Time Complexity Backward substitution O(n^3) LU method O(n^3) Space Complexity Backward substitution O(n^2) LU method O(n^2) Application Gaussian Elimination plays a major role in solving Linear systems, including: - Digital Signal processing - Linear system analysis - Image processing. - Etc…