Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
The Islamic University of Gaza Faculty of Engineering Civil Engineering Department Numerical Analysis ECIV 3306 Chapter 10 LU Decomposition and Matrix Inversion Introduction • Gauss elimination solves [A] {x} ={B} • It becomes insufficient when solving these equations for different values of {B} • LU decomposition works on the matrix [A] and the vector {B} separately. • LU decomposition is very useful when the vector of variables {x} is estimated for different parameter vectors {B} since the forward elimination process is not performed on {B}. LU Decomposition If: L: lower triangular matrix U: upper triangular matrix Then, [A]{X}={B} can be decomposed into two matrices [L] and [U] such that: 1. [L][U] = [A] ([L][U]){X} = {B} LU Decomposition Consider: [U]{X} = {D} So, [L]{D} = {B} 2. [L]{D} = {B} is used to generate an intermediate vector {D} by forward substitution. 3. Then, [U]{X}={D} is used to get {X} by back substitution. Summary of LU Decomposition LU Decomposition As in Gauss elimination, LU decomposition must employ pivoting to avoid division by zero and to minimize round off errors. The pivoting is done immediately after computing each column. LU Decomposition System of linear equations [A]{x}={B} a11 a12 a a 21 22 a31 a32 a13 x1 b1 a23 x2 b2 a33 x3 b3 Step 1: Decomposition a11 a12 a13 [U ] 0 a22/ a23/ // 0 0 a33 a12 l 21 ; a11 a31 l 31 a11 a32\ l 32 \ a22 [ L][U ] [ A] 1 0 [L ] l 21 1 l 31 l 32 0 0 1 LU Decomposition Step 2: Generate an intermediate vector {D} by forward substitution 1 0 0 d1 b1 l 21 1 0 d 2 b2 l31 l32 1 d 3 b3 Step 3: Get {X} by back substitution. a11 a12 0 a' 22 0 0 a13 x1 d1 a'23 x2 d 2 a' '33 x3 d 3 LU Decomposition-Example 3 0.1 0.2 A 0.1 7 0.3 0.3 0.2 10 0.2 3 0.1 0 7.003 0.293 0 0.19 10.02 0.1 0.3 l 21 0.03333; l 31 0.1000 3 3 a32\ 0.19 l 32 \ 0.02713 a22 7.003 3 0.1 0.2 U 0 7.003 0.293 0 0 10 . 012 0 0 1 [L ] 0.03333 1 0 0.1000 .02713 1 LU Decomposition-Example (cont’d) Use previous L and D matrices to solve the system: 3 0.1 0.2 x1 7.85 0.1 x 19.3 7 0 . 3 1 2 0.3 0.2 10 x3 71.4 Step 2: Find the intermediate vector {D} by forward substitution 0 0 d1 7.85 d1 7.85 1 0.0333 d 19.3 d 19.5617 1 0 2 2 0.1000 0.02713 1 d 3 71.4 d 3 70.0843 LU Decomposition-Example (cont’d) Step 3: Get {X} by back substitution. 0.2 x1 7.85 x1 3 3 0.1 0 7.0033 0.2933 x 19.5617 x 2.5 2 2 0 0 10.012 x3 70.0843 x3 7.00003 Decomposition Step • % Decomposition Step for k=1:n-1 [a,o]= pivot(a,o,k,n); for i = k+1:n a(i,k) = a(i,k)/a(k,k); a(i,k+1:n)= a(i,k+1:n)-a(i,k)*a(k,k+1:n); end end Partial Pivoting • %Partial Pivoting function [a,o] = pivot(a,o,k,n) [big piv]=max(abs(a(k:n,k))); piv=piv+(k-1); if piv ~= k temp = a(piv,:); a(piv,:)= a(k,:); a(k,:)=temp; temp = o(piv); o(piv)=o(k); o(k)=temp; end Substitution Steps %Forward Substitution d(1)=bn(1); for i=2:n d(i)=bn(i)-a(i,1:i-1)*(d(1:i-1))'; end • % Back Substitution x(n)=d(n)/a(n,n); for i=n-1:-1:1 x(i)=(d(i)-a(i,i+1:n)*x(i+1:n)')/a(i,i); end Matrix Inverse Using the LU Decomposition • LU decomposition can be used to obtain the inverse of the original coefficient matrix [A]. • Each column j of the inverse is determined by using a unit vector (with 1 in the jth raw ). Matrix Inverse: LU Decomposition [A] [A]-1 = [A]-1[A] = I Ax1 {b}1 Ax2 {b}2 Ax3 {b}3 1 0 0 0 1 0 0 0 1 Ax1 1st column of [A]-1 A 1 Ax2 2nd column of [A]-1 {x}1 {x}2 {x}3 Ax3 3rd column of [A]-1 Matrix inverse using LU decomposition Example 3 0.1 0.2 A 0.1 7 0.3 0.3 0.2 10 0 0 1 [L ] 0.03333 1 0 0.1000 .02713 1 3 0.1 0.2 U 0 7.003 0.293 0 0 10.012 1A. [L]{d}1 = {b}1 0 0 d1 1 d1 1 1 0.0333 d 0 d 0.03333 1 0 2 2 0.1000 0.02713 1 d 3 0 d 3 0.1009 1st column 1B. Then, [U]{X}1={d}1 of [A]-1 0.2 x1 1 3 0.1 x1 0.33249 0 7.0033 0.2933 x 0.03333 x 0.00518 2 2 0 0 10.012 x3 0.1009 x3 0.01008 Matrix inverse using LU decomposition Example (cont’d) 2A. [L]{d}2 = {b}2 0 0 d1 0 d1 0 1 0.0333 1 0 d 2 1 d 2 1 0.1000 0.02713 1 d 3 0 d 3 0.02713 2B. Then, [U]{X}2={d}2 2nd column of [A]-1 0.2 x1 0 x1 0.004944 3 0.1 0 7.0033 0.2933 x 1 x 0.142903 2 2 0 0 10.012 x3 0.02713 x3 0.00271 Matrix inverse using LU decomposition Example (cont’d) 3A. [L]{d}3 = {b}3 0 0 d1 0 d1 0 1 0.0333 d 0 d 0 1 0 2 2 0.1000 0.02713 1 d 3 1 d 3 1 3B. Then, [U]{X}3={d}3 3rd column of [A]-1 0.2 x1 0 x1 0.006798 3 0.1 0 7.0033 0.2933 x 0 x 0.004183 2 2 0 0 10.012 x3 1 x3 0.09988 Matrix inverse using LU decomposition Example (cont’d) 0.33249 0.004944 0.006798 [ A]1 0.00518 0.142903 0.004183 0.01008 0.00271 0.09988 Vector and Matrix Norms Norm is a real-valued function that provides a measure of size or “length” of vectors and matrices. Norms are useful in studying the error behavior of algorithms. A simple example is a vector in three - dimensiona l Euclidean space that can be represente d as F a b c where a, b, and c are the distances along x, y, and z axes, repectivel y. Vector and Matrix Norms (cont’d) Vector and Matrix Norms (cont’d) • The length of this vector can be simply computed as F e a 2 b2 c 2 Length or Euclidean norm of [F] • For an n dimensional vector X x1 x2 xn a Euclidean norm is computed as Xe n 2 x i i 1 For a matrix [A] Ae n n i 1 j 1 ai2, j Frobenius norm Vector and Matrix Norms (cont’d) P-Norm 1/ p X p xi i 1 1-Norm n p For Vector n X 1 xi i 1 For Matrix (Column Sum Norm) n A 1 max ai , j 1 j n i 1 Vector and Matrix Norms (cont’d) • Uniform vector norm X • max xi 1i n Uniform matrix norm (row sum Norm) n A max ai , j 1i n j 1 Vector and Matrix Norms (cont’d) • Matrix Condition umber Defined as: Cond A A A1 • For a matrix [A], this number will be greater than or equal to 1. • If the coefficients of [A] are known to t-digit precision (rounding errors~10-t) and Cond [A]=10c, the solution [X] may be valid to only t-c digits (rounding errors~10c-t). Iterative Refinement • Round-off errors can be reduced by the following procedure: a11x1 a12 x2 a13 x3 b1 a21x1 a22 x2 a23 x3 b2 a31x1 a32 x2 a33 x3 b3 • Suppose an approximate solution vectors given by ~ T ~ ~ ~ X [ x1 x2 x3 ] • Substitute the result in the original system ~ ~ ~ ~ a11 x1 a12 x2 a13 x3 b1 ~ ....(Eq.1) ~ ~ ~ a21 x1 a22 x2 a23 x3 b2 ~ ~ ~ ~ a31 x1 a32 x2 a33 x3 b3 Iterative Refinement (cont’d) • Now, assume the exact solution is: x1 ~ x1 x1 x ~ x x 2 2 2 x3 ~ x3 x3 • Then: a11 ( ~ x1 x1 ) a12 ( ~ x2 x2 ) a13 ( ~ x3 x3 ) b1 a21 ( ~ x1 x1 ) a22 ( ~ x2 x2 ) a23 ( ~ x3 … x3 ) b2 …….(Eq.2) a (~ x x ) a ( ~ x x ) a ( ~ x x ) b 31 1 1 32 2 2 33 3 3 3 Iterative Refinement (cont’d) Subtract Eq.2 from Eq.1, the result is a system of linear equations that can be solved to obtain the correction factors x ~ a11x1 a12x2 a13x3 b1 b1 E1 ~ a21x1 a22x2 a23x3 b2 b2 E2 ~ a31x1 a32x2 a33x3 b3 b3 E3 The factors then can be applied to improve the solution as ~ x specified by the equation: 1 x1 x1 x ~ x x 2 2 2 x3 ~ x3 x3 Iterative Refinement- Example (cont’d) Solve: 4.23x1 1.06 x2 2.11x3 5.28 2.53x1 6.77 x2 0.98 x3 5.22 1.85 x1 2.11x2 2.32 x3 2.58 The exact solution is ……… X T [1.00 1.00 1.00] 1- Solve the equations using [A]-1, such as {x}=[A]-1{c} ~ T X [0.991 0.997 1.00] Iterative Refinement - Example (cont’d) 2- Substitute the result in the original system [A]{x}={c} ~ ~ [ A]{ X } {C} 5.24511 ~ {C} 5.22246 2.59032 5.28 5.24511 0.0348 ~ E {C} {C} 5.22 5.22246 0.00246 2.58 2.59032 0.0103 Iterative Refinement - Example (cont’d) ~ 3- Solve the system of linear equations [ A]{X } {E} using [A]-1 to find the correction factors x to yield X T [0.00822 0.00300 0.00000757] x1 ~ x1 x1 0.999 x ~ x x 1.00 2 2 2 x3 ~ x3 x3 1.00