Download - 1 - AMS 147 Computational Methods and Applications Lecture 17

AMS 147 Computational Methods and Applications
Lecture 17
Copyright by Hongyun Wang, UCSC
Recap:
Solving linear system A x = b
Suppose we are given the decomposition,
A = L U.
We solve (LU) x = b in 2 steps:
*) Solve L y = b using the forward substitution
Cost = N 2
*) Solve U x = y using the backward substitution
Cost = N 2
Cost of the 2 steps = 2 N 2.
LU decomposition:
Cost =
2 3
N
3
The main cost of solving A x = b is in the LU decomposition.
Numerical solution xˆ can be viewed as the exact solution of a perturbed system
A x̂ = b + b
If the numerical method is bad, the perturbation may be very large
An example to demonstrate the importance of pivoting (continued)
Consider linear system
a1 x1 + x 2 = 1
x1 + a2 x 2 = b2
where
a1 = 2 64 ,
Method #1:
a2 = 1 ,
b2 = 1
Gauss elimination without pivoting
…
-1-
AMS 147 Computational Methods and Applications
xˆ1 = 0
xˆ 2 = 1
Substituting the numerical solution into the original linear system, we have
a1 x̂1 + x̂2 = 1
x̂1 + a2 x̂2 = b2 +
( 2 )
Large
perturbation
The numerical solution xˆ = ( xˆ1, xˆ 2 ) can be viewed as the exact solution of a perturbed
system. In this example, the numerical method (Gauss elimination without pivoting) is bad.
As a result, the perturbation is very large relative to the machine precision.
T
As we will see below, for a good numerical method, the perturbation is of the magnitude of
machine precision.
Method #2:
Gauss elimination with pivoting
"Pivoting" means we use the largest element to do the elimination.
Elimination:
Interchange row 1 and row 2, we have
a1
1
1 1
a2 b2 1
a
a2 b2 1 1 1
Add a1 (row 1) to (row 2)
1
b2
a2
0 fl(1 a a ) fl(1 a b ) 1 2
1 2
Let us calculate these floating point numbers.
Notice that a1 a2 = 2 64 and a1 b2 = 2 64 . We have
(
)
fl (1 a1 a2 ) = fl 1 + 2 64 = 1
(
)
fl (1 a1 b ) = fl 1 2 64 = 1
Substitution:
Row 2:
==>
fl (1 a1 a2 ) x̂2 = fl (1 a1 b2 )
x̂2 =
fl (1 a1 b )
fl (1 a1 a2 )
=
1
=1
1
-2-
AMS 147 Computational Methods and Applications
x̂1 + a2 x̂2 = b2
Row 1:
==>
xˆ1 = b2 a2 xˆ 2 = 1 (1) = 2
==>
xˆ1 = 2
xˆ 2 = 1
Substituting the numerical solution into the original linear system, we obtain
a1 x̂1 + x̂2 = 1 +
2 63
Small
perturbation
x̂1 + a2 x̂2 = b2
The numerical solution xˆ can be viewed as the exact solution of a perturbed system.
For Gauss elimination with pivoting, the perturbed system is very close to the original
system. Small perturbation is the character of good numerical methods.
Error analysis in solving A x = b
Goal: to study the effect of round-off error.
To prepare for the error analysis, we review vector norm and define matrix norm.
Vector norm:
x = ( x1 , x2 , …, x N )
T
1
p-norm:
x
1-norm:
x
p
N
p p
= xj j =1
N
= xj
1
j =1
1
2-norm:
x
-norm:
x
N
2 2
= xj j =1
2
= max x j
j
Matrix norm:
A
= max
p
x0
Ax
x
p
p
-3-
AMS 147 Computational Methods and Applications
It follows directly from the definition that
Ax
x
p
A
p
p
Ax
==>
p
A
p
x
p
We will use this result in the error analysis.
Note: The definition does not give us a direct way of calculating the matrix norm.
Fortunately we have explicit formulas for a few cases:
a11 a1N A= a
aNN N1
1-norm:
A
2-norm:
A
N
=
max
ai j
1
j i =1
2
(
= max j AT A
j
(
)
)
where j AT A denotes the j-th eigenvalue of matrix AT A .
-norm:
A
N
= max ai j i j =1
Notation for error analysis:
x:
the exact solution of A x = b
xˆ :
the numerical solution of A x = b
(the solution obtained using IEEE double precision)
Numerical solution xˆ can be viewed as the exact solution of a perturbed system
A x̂ = b + b
We write the numerical solution xˆ as
xˆ = x + x
Specific goal of the error analysis:
to relate the relative perturbation to the relative error.
-4-
AMS 147 Computational Methods and Applications
To relate
x
b
to
.
x
b
Relative
perturbation
Relative
error
We start with A x̂ = b + b .
A x̂ = b + b
==>
A ( x + x ) = b + b
==>
A x + A x = b + b
(Using A x = b )
==>
A x = b
(Multiplying by A1)
==>
==>
x = A 1 b
x = A 1 b A 1 b
(R1)
On the other hand, starting with A x = b , we have
b = Ax
==>
b = Ax A x
(Dividing by b x )
==>
1
A
x
1
b
(R2)
Combining (R1) and (R2), we obtain
x
b
1
A A x
b
Definition:
The condition number of matrix A is defined as
cond( A ) = A
1
A
In terms of the condition number, we have
-5-
AMS 147 Computational Methods and Applications
x
b
cond ( A)
x
b
Relative
error in solution
Relative
perturbation
Note:
When cond(A) is large (for example, cond ( A ) 1014 ), we say matrix A is ill-conditioned.
When cond(A) is small (for example, cond ( A ) 10 6 ), we say matrix A is well-conditioned.
Two causes for large error
•
x
:
x
b
~ O(1)
b
If the numerical method is bad, then the relative perturbation is large. Consequently, the
relative error in the solution is large.
Remedy: use a good numerical method.
•
cond( A ) ~ O
1
If the system is ill conditioned, then the relative error in the solution is large even when
we use a good numerical method.
Two remedies:
1) use a higher precision
2) go back to the modeling process to formulate a well-conditioned system
A geometric view an ill-conditioned system:
[Draw two nearly perpendicular lines to show a well-conditioned system]
[Draw two nearly parallel lines to show an ill-conditioned system]
Solving sparse linear systems
Au = b
where A is an N N matrix satisfying
-6-
AMS 147 Computational Methods and Applications
•
•
•
6
N is large (for example, N = 10 );
Each row of A has only a few non-zero elements (for example, at most, 5 non-zero
elements);
A w can be calculated efficiently without using the matrix form of A.
Example:
The boundary value problem of 1-D Poisson equation:
u x x = b ( x )
u ( 0 ) = 0 , u (1) = 0
The numerical grid
We divide [0,1] into n subintervals.
h=
1
,
n
xi = i h ,
i = 0, 1, …, n
Recall the numerical differentiation method for approximating the second derivative
f ( x ) f ( x h ) + f ( x + h) 2 f ( x )
h2
Applying this to u x x = b ( x ) , we obtain
The numerical discretization:
ui1 2 ui + ui+1
h2
= bi ,
i = 1, 2, …, n 1
where
bi = b ( xi )
ui = numerical approximation of u ( xi ) .
The boundary condition:
u0 = 0, un = 0
The numerical discretization has the form of a linear system
Au = b
where
u = uj , 1 j n 1
b = bj , 1 j n 1
{
{
}
}
-7-
AMS 147 Computational Methods and Applications
A is an N N matrix satisfying
•
•
•
N = (n – 1), number of elements in u
Each row of A has at most 3 non-zero elements
For any vector w = w j , 1 j n 1 ,
A w is calculated efficiently without using the matrix form of A
{
( A w )i =
}
wi 1 2wi + wi +1
h2
,
1 i n 1
w0 = wn = 0
Note: For the 1-D problem, it is actually easy to write out the matrix form of A.
2 1
1 2 1
1
1 2 A= 2
h 2 1 1 2 But working with the matrix form of A is not viable in 2-D or 3-D problems where the matrix
form of A is very complicated.
The boundary value problem of 2-D Poisson equation:
ux x + uy y = b( x,y )
in [0,1] [0,1]
u( x,y ) = 0
on the boundary of [0,1] [0,1]
[Draw the square to show the computational domain and the boundary].
The numerical grid
We divide [0,1] into n subintervals.
h=
1
,
n
xi = i h ,
i = 0, 1, …, n
yj = j h ,
j = 0, 1, …, n
The numerical discretization:
ui 1 j 2 ui j + ui +1 j
h
2
+
ui j 1 2 ui j + ui j +1
h2
= bi j ,
where
(
bi j = b x i, y j
)
-8-
1 i, j n 1
AMS 147 Computational Methods and Applications
(
)
ui j = numerical approximation of u x i , y j .
The boundary condition:
u0 j = 0, un j = 0, ui 0 = 0, ui n = 0 ,
1 i, j n 1
The numerical discretization has the form of a linear system
Au = b
where
{
{
}
}
u = ui j , 1 i, j ( n 1)
b = bi j , 1 i, j ( n 1)
A is an N N matrix satisfying
•
•
•
N = (n 1)2, number of elements in u .
Each row of A has at most 5 non-zero elements.
For any vector w = wi j , 1 i, j ( n 1) ,
A w is calculated efficiently without using the matrix form of A
{
( A w )i j =
}
wi 1 j + wi +1 j + wi j 1 + wi j +1 4 wi j
h2
w0 j = 0, wn j = 0, wi 1 = 0, wi n = 0 ,
-9-
,
1 i, j ( n 1)
1 i, j n 1