* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Handout #5
Survey
Document related concepts
Linear least squares (mathematics) wikipedia , lookup
Rotation matrix wikipedia , lookup
Determinant wikipedia , lookup
Matrix (mathematics) wikipedia , lookup
System of linear equations wikipedia , lookup
Four-vector wikipedia , lookup
Orthogonal matrix wikipedia , lookup
Jordan normal form wikipedia , lookup
Non-negative matrix factorization wikipedia , lookup
Singular-value decomposition wikipedia , lookup
Principal component analysis wikipedia , lookup
Matrix calculus wikipedia , lookup
Gaussian elimination wikipedia , lookup
Eigenvalues and eigenvectors wikipedia , lookup
Perron–Frobenius theorem wikipedia , lookup
Transcript
Handout #5 Matrix and Von Neumann Stability Analyses (Continued…) In the previous handout, we showed that the error in the solution at the end of an iteration (n+1) is related to the error at the start of the iteration by the relation: en1 M n1 e0 (1) where the matrix {M} is given by: M I B 1 A (2) Recall that [A] is the original matrix for solving the system AT b (3) while [B] is a more easily inverted matrix, which allows us to solve the system: BT b AT n R (4) Here {R} is called the residual. If [B] is identical to [A], then [M] will be a null matrix( i.e. a matrix with only zero elements). In that case, the error (as given by equation 1) will reduce to zero in just one iteration. This is not surprising, because if [B] equals [A], we are solving equation (3) exactly. We can also conclude that the closer is the matrix [B] to the original matrix [A], the faster it will converge. This is why an SLOR scheme which retains 4 of the original 5 diagonals of the matrix [A] in your computer project works so well compared to the Point Jacobi scheme, which retains only the main diagonal of [A]. Using equation (1) to study the stability of iterative schemes is not practical, because it requires computing the matrix [M], which is relatively easily done, and multiplying [M] by itself several times (not so easily done). This effort is necessary to make sure that[M]n+1 turns into a null matrix after sufficiently large number of ‘n’, i.e. after lots of iterations. Fortunately, there is an easier way. From matrix algebra, any matrix [M] may be written as M T T 1 (5) where {] is a diagonal matrix which contains the eigenvalues of [M], while T contains the eigenvectors of the matrix [M] as its columns. If you have forgotten what eigenvalues and eigenvectors are, join the club. Eigenvalues are simply solutions to the equation: detM I 0 (6) If [M] is an NxN matrix, there will be N roots. These will, in general, be complex numbers. A program such as Mathematica or MATLAB will spit out these eigenvalues in a matter of seconds (unless you use a Mac) for sufficiently small values of N. An eigenvector is an array {x} that is a solution to: M x x (7) while MATLAB will give the eigenvectors as well, these are not needed. Using equation (5), we can write [M]n+1 as: M n1 [M ][ M ][ M ].... [T ][][T ]1[T ][][T ]1[T ][][T ]1 ... [T ][]n1[T ]1 (8) If the absolute magnitude of each of the eigenvalues of [] were less than one (unity for you mathematicians), then []n+1 will go to zero. After all, multiplying a number less than unity (real or complex) by itself enough times will yield zero. In summary, “If the absolute value of the largest eigenvalue of the matrix [M]= [I-B-1A] is less than unity, the iterative process will converge. “ (Note: The absolute value of the largest eigenvalue of a matrix is called the spectral radius of the matrix by mathematicians). Application of the matrix stability analysis is easy, although it consumes a lot of memory and can only be done on a computer. It requires (a) computing the matrix [A], (b) computing the matrix [B], (c) computing the matrix [M], and, (d) finally computing the spectral radius of [M]. Matrix stability analysis is usually done for a small size matrix (one corresponding to N=5 or N=10), say. If it works for such small size problems, it is assumed (with the fingers crossed) that the spectral radius will be less than unity for large size problems as well. Of course, one can arbitrarily try different [B] matrices in an iterative scheme to see which one works. This kind of programming seldom works (although there have been a few brilliant discoveries totally attributable to pure chance- e.g. penicillin, radium). As Robert Heinlein says in his sci-fi classic Strangers in a Strange Land, “Experience is a hard teacher. But fools will learn no other way…” Von Neumann Stability Analysis The Von Neumann stability analysis is another way of finding which choices for the matrix [B] work best. We will look at the stability of Point Jacobi scheme as predicted by the Von Neumann stability analysis. Other schemes (SOR, Gauss-Seidel etc.) can be analyzed in a similar fashion. For the Point Jacobi scheme, from equation (12) of handout #4, at a typical node i, we get: 2 Ti n 1 Ti n Ti n1 2Ti n Ti n1 q x 2 x 2 (9) The exact solution to our discretized form of the equation is: 2 Ti exact Ti exact Ti exact 2Ti exact Ti exact 1 1 q x 2 x 2 (10) Subtract (10) from equation (9). The difference between the computed solution at any iteration‘n’ (or ‘n+1’) and the exact solution is given the symbol ‘e’. In other words, T n T exact e n T n 1 T exact e n 1 (11) With this definition of error, equations (10) subtracted from equation (9) gives: 2 ein 1 ein ein1 2ein ein1 x 2 x 2 (12) Simplifying equation (12), we get for the Point Jacobi scheme: ein 1 ein1 ein1 2 (13) Von Neumann assumed that the error in a solution at the end of any iteration ‘n’ may be written as a Fourier series: e n x Apn exp( jpx) e n1 x Apn 1 exp( jpx) (14) This may not look like the Fourier series that we learned (and forgot) in Math 4582. The symbol j is simply the square root of –1. This quantity is usually called ‘i’ in complex number theory, but we have already used the symbol ‘i’ to denote the nodes. The quantity exp(jpx) is simply sin(px) + j cos(px). The coefficient Apn is a complex number of the type C+jD where C and D are real numbers. If we perform the multiplication of Apn and exp(jpx), and retain only the real part, we will obtain a more traditional representation of the Fourier series. The symbol ‘p’ is called the wave number. In the Fourier series above, the terms for which ‘p’ is small represents slowly varying terms (sin2x, for example), where as large values of p (sin10x for example) represents rapidly varying components. The components corresponding to small values of p are called, therefore, the low frequency error components, and vice versa. The quantity Apn is called the amplitude of the component exp(jpx) at the iteration n. Obviously, the smaller this amplitude, the smaller the error. We hope that as the iterations increase Apn will go to zero for each and every component in the above series. In other words, we hope that the ratio Apn+1 / Apn is less than 1 in magnitude for all ‘n’ (i.e. for every iteration) for all ‘p’. Because equation (13) is linear, Von Neumann suggested that we can test one component of the Fourier series at a time to see how if that particular component varies. Taking a typical term in the Fourier series given in equation (14) and plugging it into equation (13), we get: A pn exp( jp[ x x]) A pn exp( jp[ x x]) A pn 1 exp( jpx) 2 Cancel _ common _ terms : A n 1 p A n 1 p A pn A pn exp( j[ px]) A pn exp( j[ px]) 2 A pn cos( px) cos( px) (15) The Amplitude at the end of the n+1 iteration of a component of the error will always be less than the amplitude at the end of iteration n for that component. In other words, Point Jacobi method will always converge, since cos(px) is always less than unity. Of course, if p is small, or x is small (fine grid), the convergence will be extremely slow. This is why your computer code converges more slowly on a fine grid than on a coarse grid. We will next apply the Von Neumann stability analysis to the Gauss Seidel scheme. From handout #4, for our model ODE, Gauss-Seidel scheme becomes: Ti 1 Ti n1 2Ti n Ti n1 2 T q i x 2 x 2 x 2 (16) The exact solution satisfies: Ti exact 2Ti exact Ti exact 1 1 0q 2 x (17) Subtract (17) from (16). Use: e n T n T exact e n 1 T n 1 T exact e n 1 e n T n 1 T n T We get: ein11 ein1 2 ein1 ein ein1 2ein ein1 Or , ein11 2 ein1 ein1 The error e is made of a Fourier series, given by equation (14). We test a single component of error at a time. A single component of en is Apn exp(jpx), while a single component of en+1 is given by Apn exp(jpx). Plugging these definitions in, and canceling out the common term exp(jpx), we get: Apn1 exp( px) 2 Apn1 Apn exp( px) Simplifying, we get : A pn 1 A n p exp( px) 2 exp( px)) Or , A pn 1 A n p 1 5 4 cos px Since the denominator is always greater than the numerator, the amplitudes steadily decrease. Thus the Gauss-Seidel scheme also always converges. We can similarly test other schemes (SOR, SLOR) and show they are all stable.