* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download - 1 - AMS 147 Computational Methods and Applications Lecture 17
Survey
Document related concepts
Determinant wikipedia , lookup
Jordan normal form wikipedia , lookup
Matrix (mathematics) wikipedia , lookup
Eigenvalues and eigenvectors wikipedia , lookup
Four-vector wikipedia , lookup
Linear least squares (mathematics) wikipedia , lookup
Perron–Frobenius theorem wikipedia , lookup
Cayley–Hamilton theorem wikipedia , lookup
Singular-value decomposition wikipedia , lookup
Orthogonal matrix wikipedia , lookup
Non-negative matrix factorization wikipedia , lookup
Matrix calculus wikipedia , lookup
Matrix multiplication wikipedia , lookup
Transcript
AMS 147 Computational Methods and Applications Lecture 17 Copyright by Hongyun Wang, UCSC Recap: Solving linear system A x = b Suppose we are given the decomposition, A = L U. We solve (LU) x = b in 2 steps: *) Solve L y = b using the forward substitution Cost = N 2 *) Solve U x = y using the backward substitution Cost = N 2 Cost of the 2 steps = 2 N 2. LU decomposition: Cost = 2 3 N 3 The main cost of solving A x = b is in the LU decomposition. Numerical solution xˆ can be viewed as the exact solution of a perturbed system A x̂ = b + b If the numerical method is bad, the perturbation may be very large An example to demonstrate the importance of pivoting (continued) Consider linear system a1 x1 + x 2 = 1 x1 + a2 x 2 = b2 where a1 = 2 64 , Method #1: a2 = 1 , b2 = 1 Gauss elimination without pivoting … -1- AMS 147 Computational Methods and Applications xˆ1 = 0 xˆ 2 = 1 Substituting the numerical solution into the original linear system, we have a1 x̂1 + x̂2 = 1 x̂1 + a2 x̂2 = b2 + ( 2 ) Large perturbation The numerical solution xˆ = ( xˆ1, xˆ 2 ) can be viewed as the exact solution of a perturbed system. In this example, the numerical method (Gauss elimination without pivoting) is bad. As a result, the perturbation is very large relative to the machine precision. T As we will see below, for a good numerical method, the perturbation is of the magnitude of machine precision. Method #2: Gauss elimination with pivoting "Pivoting" means we use the largest element to do the elimination. Elimination: Interchange row 1 and row 2, we have a1 1 1 1 a2 b2 1 a a2 b2 1 1 1 Add a1 (row 1) to (row 2) 1 b2 a2 0 fl(1 a a ) fl(1 a b ) 1 2 1 2 Let us calculate these floating point numbers. Notice that a1 a2 = 2 64 and a1 b2 = 2 64 . We have ( ) fl (1 a1 a2 ) = fl 1 + 2 64 = 1 ( ) fl (1 a1 b ) = fl 1 2 64 = 1 Substitution: Row 2: ==> fl (1 a1 a2 ) x̂2 = fl (1 a1 b2 ) x̂2 = fl (1 a1 b ) fl (1 a1 a2 ) = 1 =1 1 -2- AMS 147 Computational Methods and Applications x̂1 + a2 x̂2 = b2 Row 1: ==> xˆ1 = b2 a2 xˆ 2 = 1 (1) = 2 ==> xˆ1 = 2 xˆ 2 = 1 Substituting the numerical solution into the original linear system, we obtain a1 x̂1 + x̂2 = 1 + 2 63 Small perturbation x̂1 + a2 x̂2 = b2 The numerical solution xˆ can be viewed as the exact solution of a perturbed system. For Gauss elimination with pivoting, the perturbed system is very close to the original system. Small perturbation is the character of good numerical methods. Error analysis in solving A x = b Goal: to study the effect of round-off error. To prepare for the error analysis, we review vector norm and define matrix norm. Vector norm: x = ( x1 , x2 , …, x N ) T 1 p-norm: x 1-norm: x p N p p = xj j =1 N = xj 1 j =1 1 2-norm: x -norm: x N 2 2 = xj j =1 2 = max x j j Matrix norm: A = max p x0 Ax x p p -3- AMS 147 Computational Methods and Applications It follows directly from the definition that Ax x p A p p Ax ==> p A p x p We will use this result in the error analysis. Note: The definition does not give us a direct way of calculating the matrix norm. Fortunately we have explicit formulas for a few cases: a11 a1N A= a aNN N1 1-norm: A 2-norm: A N = max ai j 1 j i =1 2 ( = max j AT A j ( ) ) where j AT A denotes the j-th eigenvalue of matrix AT A . -norm: A N = max ai j i j =1 Notation for error analysis: x: the exact solution of A x = b xˆ : the numerical solution of A x = b (the solution obtained using IEEE double precision) Numerical solution xˆ can be viewed as the exact solution of a perturbed system A x̂ = b + b We write the numerical solution xˆ as xˆ = x + x Specific goal of the error analysis: to relate the relative perturbation to the relative error. -4- AMS 147 Computational Methods and Applications To relate x b to . x b Relative perturbation Relative error We start with A x̂ = b + b . A x̂ = b + b ==> A ( x + x ) = b + b ==> A x + A x = b + b (Using A x = b ) ==> A x = b (Multiplying by A1) ==> ==> x = A 1 b x = A 1 b A 1 b (R1) On the other hand, starting with A x = b , we have b = Ax ==> b = Ax A x (Dividing by b x ) ==> 1 A x 1 b (R2) Combining (R1) and (R2), we obtain x b 1 A A x b Definition: The condition number of matrix A is defined as cond( A ) = A 1 A In terms of the condition number, we have -5- AMS 147 Computational Methods and Applications x b cond ( A) x b Relative error in solution Relative perturbation Note: When cond(A) is large (for example, cond ( A ) 1014 ), we say matrix A is ill-conditioned. When cond(A) is small (for example, cond ( A ) 10 6 ), we say matrix A is well-conditioned. Two causes for large error • x : x b ~ O(1) b If the numerical method is bad, then the relative perturbation is large. Consequently, the relative error in the solution is large. Remedy: use a good numerical method. • cond( A ) ~ O 1 If the system is ill conditioned, then the relative error in the solution is large even when we use a good numerical method. Two remedies: 1) use a higher precision 2) go back to the modeling process to formulate a well-conditioned system A geometric view an ill-conditioned system: [Draw two nearly perpendicular lines to show a well-conditioned system] [Draw two nearly parallel lines to show an ill-conditioned system] Solving sparse linear systems Au = b where A is an N N matrix satisfying -6- AMS 147 Computational Methods and Applications • • • 6 N is large (for example, N = 10 ); Each row of A has only a few non-zero elements (for example, at most, 5 non-zero elements); A w can be calculated efficiently without using the matrix form of A. Example: The boundary value problem of 1-D Poisson equation: u x x = b ( x ) u ( 0 ) = 0 , u (1) = 0 The numerical grid We divide [0,1] into n subintervals. h= 1 , n xi = i h , i = 0, 1, …, n Recall the numerical differentiation method for approximating the second derivative f ( x ) f ( x h ) + f ( x + h) 2 f ( x ) h2 Applying this to u x x = b ( x ) , we obtain The numerical discretization: ui1 2 ui + ui+1 h2 = bi , i = 1, 2, …, n 1 where bi = b ( xi ) ui = numerical approximation of u ( xi ) . The boundary condition: u0 = 0, un = 0 The numerical discretization has the form of a linear system Au = b where u = uj , 1 j n 1 b = bj , 1 j n 1 { { } } -7- AMS 147 Computational Methods and Applications A is an N N matrix satisfying • • • N = (n – 1), number of elements in u Each row of A has at most 3 non-zero elements For any vector w = w j , 1 j n 1 , A w is calculated efficiently without using the matrix form of A { ( A w )i = } wi 1 2wi + wi +1 h2 , 1 i n 1 w0 = wn = 0 Note: For the 1-D problem, it is actually easy to write out the matrix form of A. 2 1 1 2 1 1 1 2 A= 2 h 2 1 1 2 But working with the matrix form of A is not viable in 2-D or 3-D problems where the matrix form of A is very complicated. The boundary value problem of 2-D Poisson equation: ux x + uy y = b( x,y ) in [0,1] [0,1] u( x,y ) = 0 on the boundary of [0,1] [0,1] [Draw the square to show the computational domain and the boundary]. The numerical grid We divide [0,1] into n subintervals. h= 1 , n xi = i h , i = 0, 1, …, n yj = j h , j = 0, 1, …, n The numerical discretization: ui 1 j 2 ui j + ui +1 j h 2 + ui j 1 2 ui j + ui j +1 h2 = bi j , where ( bi j = b x i, y j ) -8- 1 i, j n 1 AMS 147 Computational Methods and Applications ( ) ui j = numerical approximation of u x i , y j . The boundary condition: u0 j = 0, un j = 0, ui 0 = 0, ui n = 0 , 1 i, j n 1 The numerical discretization has the form of a linear system Au = b where { { } } u = ui j , 1 i, j ( n 1) b = bi j , 1 i, j ( n 1) A is an N N matrix satisfying • • • N = (n 1)2, number of elements in u . Each row of A has at most 5 non-zero elements. For any vector w = wi j , 1 i, j ( n 1) , A w is calculated efficiently without using the matrix form of A { ( A w )i j = } wi 1 j + wi +1 j + wi j 1 + wi j +1 4 wi j h2 w0 j = 0, wn j = 0, wi 1 = 0, wi n = 0 , -9- , 1 i, j ( n 1) 1 i, j n 1