Download Lecture: 9

Lecture: 9
BASES of NUMERICAL SOLUTION of ORDINARY
DIFFERENTIAL EQUATIONS (ODE)
The lecture discusses theoretical problems of solving ODE and covers Eulers, RungeKutta and predictor-corrector methods. Also the lecture presents an introduction to
stiff equation's problem.
INPUT PROBLEM and EXAMPLES
1.
Statement of the problem
Consider
du( t )
Cauchy problem for the system of ODE
dt = f(t,u),
t> 0, u(0) = u(0)
(1)
or more explicitly
dui( t )
dt = fi(t,u1, u2,..., um ), t > 0, i=1, 2, ..., m,.
(2)
ui(0) = ui(0),
i=1,2, ..., m.
(3)
Conditions guaranteeing the existence and uniqueness of the solution of
Cauchy problem are well known. Suppose that functions fi , i=1,2, ..., m,
are continues with respect to all arguments on the closed domain
D = {|t| a, |ui - ui(0)| b, i=1,2,...m}.
From continuity of functions fi it follows their boundedness, that is,
existence of the constant M > 0 such that inequalities | fi |  M, i=1,2 ,...,
m are valid everywhere on D.
Suppose, in addition that functions fi satisfy Lipschitz condition with
respect to all arguments u1, u2,..., um , e.g. ,
|fi(t,u’1, u’2,..., u’m ) - fi(t,u’’1, u’’2,..., u’’m )|  L{ |u’1 - u’’1| + |u’2 - u’’2| + ...
|u’m - u’’m| }
for any points t,u’1, u’2,..., u’m and t,u’’1, u’’2,..., u’’m in the domain D.
If formulated above conditions are fulfilled then there is a unique
solution of the system (2)
u1= u1(t), u2= u2(t), ..., um= um(t).
This solution is defined for |t| t0 = min(a, b/M) and takes the given
initial value (3) for t = 0. So we suppose the solution exists, is unique
and has needed smoothness.
2. Examples of numerical methods
There are two groups of solutions : multystep difference methods and
Runge-Kutta methods.
For simplicity consider one equation
du( t )
dt
=
f(t,u),
t
>
0,
u(0)
=
u
(4)
0.
Here, it is assumed that we are seeking evenly spaced approximations on
some interval. Consider a set of points = {tn = n , n= 0,1,2,...},
where  > 0 is a uniform step.
Let us define u(t) as an exact solution of the equation (4), and yn = y(tn)
as an approximate solution. Note that the approximate solution is a grid
function, that is this solution is defined only in the grid nodes.
Example 1. Euler’s method.
Equation (4) can be considered as
a difference equation
(yn + 1 - yn) /  - f(tn,yn) = 0, n=0,1,2,.., y0 = u0.
(5)
When we deal with the numerical solution the main question is the
convergence of a method. Let us
fix a point t . We have a sequence of
grids  so that 0 and tn = n = t can be created. A method is
convergent if |yn + 1 - u(tn)| 0 for 0 and tn = t .
As it is usual in numerical analysis,
the main question is how to
measure the accuracy and efficiency of an approximation scheme. One
very useful characterization of a method is its order. We will say that a
method is order p if there is a constant C , independent of the stepsize 
such that
|yn + 1 - u(tn)| < C p.
We have error zn = yn -u(tn).
Since from the equation (5)
(zn+1 - zn)/ = f(tn,un + zn) - (un+1 - un)/
we can write
(6)
the right part of the equation (6) can be represented as the sum
(1)n + (2)n,
where
(1)n = - (un+1 - un)/ + f(tn,un) ,
(2)n = f(tn,un + zn) -f(tn,un) .
The function (1)n is
a residual function or an approximation error.
Generally, the difference method approximates the initial differential
equation if (1)n 0 for 0. The difference method has p-order
approximation, if (1)n < Cp.
The function (2)n vanishes,
if the right part does not depend on the
solution u, and in a common case (2)n is proportional to
zn.
Heuristic confirmation that Euler’s method is order one is provided by
Taylor’s theorem. We have for any tn
(un+1 - un)/ = u’(tn) + O(),
then since the equation (4)
(1)n =- u’(tn) + f(tn,un) + O() = O(),
Euler’s method is order one.
Example 1.1. Solve
y = -20y +7 exp(-0.5), y(0) = 5,
using the forward Euler method with  = 0.01 for 0 < t  0.1
Solution. The first few steps of calculations are shown next:
t0 = 0,
y0 = y(0) = 5
t1 =0.01
y1 = y0 + y0= 5 +(0.01)(-20(5) + 7 exp (0)) = 4.07
t1 =0.02
y2 = y1 + y1= 4.07 + (0.01)(-20(4.07) + 7 exp (-0.005)) =
= 3.32565
The numerical methods to solve first-order differential equations can be
easily applied to higher-order ODEs because higher-order ODES can be
decomposed to a set of first-order differential equations. For example ,
consider
y + a y + b y + c y + ey = g
where a,b,c,d,e, and g are constants. The initial conditions are given as
y(0) = y0, y(0) = y0, y(0) = y0, y(0) = y0
By defining u,v, and w as
u = y, v = y, w = y
it is equivalent to the set of four-order ordinary differential equations:
y = u,
y(0) = y0
u = v,
u(0) = y0
v = w,
v(0) = y0,
w = g - aw - bv - cu - ey,
w(0) = y0
Example 2. Symmetric scheme.
The equation (4) is replaced by the difference equation
(yn+1 - yn)/i - (f(tn,yn)
+ f(tn+1,yn+1))/2 = 0, .n = 0,1,...,
y0 = u0.
(7)
This method is an implicit method because the new value y n+1 is defined
from the solution of the following equation
yn+1 - 0.5 f(tn+1,yn+1) = Fn,
where Fn = yn + 0.5 f(tn,yn). An advantage of this method is confined in
more high order of approximation accuracy.
For the residual function
(1)n = -(un+1 - un)/ + (f(tn,yn)
+ f(tn+1,yn+1))/2
we have the expansion
(1)n =- u’n -  u’’n/2 + O(2) + (u’n + u’n+1)/2 =
(u’n + u’n+1+
- u’n -  u’’n/2 +
u’’n +O(2))/2,
that is (1)n = O(2). Thus the method is order two.
Example 3. Runge-Kutta method
Suppose we know an approximated solution yn of initial problem for
t = tn. To find yn+1 = y(tn+1) at the beginning Euler’s scheme
(yn + 1/2 - yn) / 0.5 = f(tn,yn)
(8)
for calculating the intermediate value yn
+ 1/2
can be used. Then the
difference equation
(yn + 1/2 - yn) / = f(tn +0.5 , yn+1/2)
(9)
is used to define yn+1 explicitly.
To investigate the residual (1)n we substitute the intermediate value yn +
1/2
= yn + 0.5 fn into the equation (9). We obtain the difference equation
(yn + 1 - yn) / - f(tn +0.5 , yn+0.5fn)=0
for which the
(10)
residual (1)n is equal
(1)n = -(un+1-un)/+f(tn+0.5,un+0.5f(tn,un)),
(11)
We derive
(un+1-un)/=un+0.5u’’n+O(2),
f(tn+0.5,un+0.5f(tn,un))=
=f(tn,un)+0.5(f(tn,un)/t+0.5f(tn,un)f(tn,un)/u)=
=f(tn,un)+ 0.5u’’n,
since from the equation (4) we have
d2u/dt2=f/ t+f f/u.
Thus the method (10) has the order two of error approximation and this
method is explicit. This two-step scheme(8) and (9) is known as a
predictor-corrector scheme. At the first step (8) we have prediction with
the accuracy O(), and at the second step we correct the predicted value.
The method (10) can be realized in another way. At first we calculate
functions
k1 = f(tn,un), k2 = f(tn+ 0.5,yn + 0.5k1),
and then define yn+1 from the equation (yn-1 - yn)/ = k2.
Such form of realization is called Runge-Kutta method and requires two
intermediate calculations of k1 and k2 .
STABILITY and STIFF DIFFERENTIAL EQUATIONS
In many applications the solution to the initial value problem
x = f(t,x), x(t0) = x0
(1)
approaches an equilibrium point x = x-, that is, a point for which f(t,x-) 
0 for all t. Approximating solutions by numerical means as they approach
an equilibrium point might be thought to be routine, but this is not
always the case, as the following example shows.
Example 1. The solution x(t) = e-t*t to the initial value problem
x = - 2tx , x(0) = 1
(2)
approaches the equilibrium point x = 0. Figure 1 shows the Euler method
approximation for stepsize of = 0.2. It illustrates an instability that is
encountered with many fixed-step numerical methods for solving
differential equations.
Figure 1.
Stability of Euler’s Method
To see more about the origin of such instability, let us consider a simpler
problem, the application of Euler’s method to the initial value problem
x = x , x(0) = x0 ,
(3)
where is constant. For Euler’s method y i + 1 =yi + yi= (1+ )yi
Thus
y0 =x0 ,
y1 = (1+ )x0 ,
y2 = (1+ )2x0 ,
..........................
(4)
yi = (1+ )ix0 .
When  < 0 , the actual solution x(t) =x0 e
t
approaches the equilibrium
point x = 0. The Euler method approximation yi = (1+ )ix0 also
approaches
x = 0 only if |1 +| < 1, that is,   (-2,0). When 1 +  <
-1, the Euler method approximation will exhibit growing oscillations
about x =0. Figure 2 shows the Euler method approximation to the
solutions of x’ = -5x , x(0) = 1 when = 0.25 and when  = 0.5.
Figure 2.
The analysis of the stability of Euler’s method for the initial value
problem (3) leads us to expect that for the problem (2), Euler’s method
will be stable so long as -2t  (-2,0).With  = 0.2 this gives t (0,5).
More generally, suppose that the solution to the initial value problem (1)
approaches the equilibrium point x =0. Then from Taylor’s theorem
f(t,x) = f(t,0) + f(t,0)/x
+ O(|x|2) = f(t,0)/x
+ O(|x|2)
(5)
since f(t,0) = 0. The initial value problem
x= f(t,0)/x,
x(t0) = x0
is referred to as the linearized form of (1). When f(t,0)/ x  0 Euler’s
method will be stable when f(t,0)/ x is in the interval of stability (-2,0)
for the problem (3).
Based upon the above considerations, we make the following definition.
Definition. The
interval of absolute stability of a numerical method is
the interval of  in which the approximation yi to the solution
x(ti) =x0 e  to (3) approaches zero as i  .
Stability of Runge-Kytta Methods
The interval of absolute stability of other Runge-Kytta Methods can be
found in a similar manner to that used for Euler’s method.
Example 2. Let us find the interval of absolute stability for the modified
Euler method
yi + 1 = yi +0.5 [f(ti,yi) + f(ti + ,yi + f(ti,yi))] .
With
f(ti,yi) = yi,
we have
y i + 1 =( 1 +   +0.52 2)yi whence
y i + 1 =( 1 +   +0.52 2)x0.
Thus the approximations approach x = 0 when | 1 +  
Since 1 +  
+0.522 | < 1.
+0.522 = 0.5( +1)2 +0.5, this implies that the interval
of absolute stability for the modified Euler method is    (-2,0), as it
was for Euler’s method. We call a method absolutely stable if it is stable
for any .
From the general formula for Runge-Kutta method, it is clear that
application of the method to the problem (3) will lead to an equation of
the form y
i+1
=P(-)yi, where - = . For the Euler and modified Euler
methods we have found P(-) = 1 + -and P(-) = 1 + -+0.5-2 ,
respectively. Since y
i+1
=P(-)yi implies y
i+1
=P(-)ix0 , the condition
for absolute stability is |P(-)| < 1.
Stiff Systems of Differential Equations
To see the implementations of the above analysis let us consider the
problem of approximating the solution to the initial value problem
x + (b +1) x’ + bx =
0, x(0) = 1, x(0) =0,
where b > 0. The solution to this problem is easily seen to be
x(t) =[b/(b-1)]e-t - [1/(b-1)]e-bt.
Problem (6) is equivalent to the initial value problem
(6)
x  1 = x2 ,
x2 = -bx1 - (b + 1)x2,
x 1(0) = 1 ,
x2(0) = 0
(7)
for first-order systems. The solution to (7) is
x1(t) = [b/(b-1)]e-t - [1/(b-1)]e-bt,
x2(t) = -[b/(b-1)]e-t + [b/(b-1)]e-bt.
(8)
Now suppose that b is much large than 1. Table 1 shows the interval of
absolute stability for Runge-Kutta methods with global truncation errors
of order one through four.
Order
P(-)
Interval
---------------------------------------------------------------------1
1 + -
(-2,0)
2
1 + -+0.5 -2
(-2,0)
3
1 + -+0.5 -2 + 1/6 -3
(-2.51,0)
4
1 + -+0.5 -2 + 1/6 -3 + 1/24 -4
(-2.78,0)
Table 1. Intervals of absolute stability for Runge-Kutta methods
Then from Table 1 a fourth-order Runge-Kutta method is stable for
stepsizes for which - < 2.78/b. If b =1000, for instance, then a stepsize of
- < 0.00278 is required.
This is far smaller than is required to keep the local truncation error of
the method within all but the most stringent tolerances. Indeed, an
implementation of the classical Runge-Kutta method in Turbo Pascal
with a stepsize of - = 0.0025 produces approximations to the solution (8)
that are about the size of the machine unit  = 2 10-12. However, if the
stepsize is increased to - = 0.003 , then the method is unstable and
floating-point overflow quickly results.
A system of differential equations such as (6) with large b, in which the
stepsize is constrained by the interval of absolute stability rather than the
need to keep the truncation error small, is said to be stiff. Methods of
numerical approximation such as the Runge-Kutta-Felberg and Gragg
methods that efficiently control the local truncation error tend to do
poorly on stiff problems also.
Obviously the notation of stiffness is very imprecise, depending as it
does upon the interplay of the truncation error and the interval of
absolute stability of a numerical method. About all that can be said
generally is that it arises when the time intervals over which different
“components” of a solution undergo changes of comparable magnitude
are very different. For instance, in the solution (8) one component decays
as e-t while the other decays as e-tb, which will be much faster if b is
large. A nonhomogeneous linear system x’(t) = Ax +f(t) could stiff either
because the eigenvalues of A differ considerably from each other or
because the transient solution decays on a time scale very different from
that appropriate for measuring changes in the steady state solution.
Approximation of Solutions to Stiff Differential Equations
A good approach to solving stiff equations is to use a method that is
absolutely stable and thus whose interval of stability cannot become a
constraint. The implicit trapezoid rule is such a method. We can
interpolate the points (ti,x(ti)) and (ti+1,x(ti+1)) rather then (ti-1,x(ti-1)) and
(ti,x(ti)). Then the method of interpolation
yi+1 =yi + 0.5 [f(ti,yi +f(ti+1,yi+1)]
is the implicit trapezoid method.
Applying the implicit trapezoid rule to the initial value problem
x = f(t,x), x(t0) = x0
(9)
gives
G(ti,yi,yi+1;) = yi+1 - yi - 0.5 [f(ti,yi) + f(ti+1,yi+1)] = 0
(10)
For known ti and yi this equation must be solved for the approximation
yi+1 to x(ti+1).
It turns out that for stiff systems simple iteration will not always
converge, and thus Newton’s method for systems must be used. Recall
that the Newton iteration for solving the system F(x) = 0 is
xi+1 = xi -{J[F](xi)}-1F(xi) ,
where J[F](x) is the Jacobin matrix of F,
|F1(x)/x1......F1(x)/xn|
J[F](x):=
|.........................................|
(11)
|Fn(x)/x1......Fn(x)/xn|
The Jacobean matrix of the function G(ti,yi,yi+1;) is easily seen to be
J[G](yi+1)= I - 0.5 J[f(ti + ,.)](yi+1). Mentioned below algorithm
outlines a procedure for approximating x(tf) using N steps of implicit
trapezoid rule.
Algorithm. Fixed-step implicit trapezoid method
Input
t0,x0 (initial conditions)
tf (point at which to find approximation)
N (number of subintervals)
tol( error tolerance for Newton’s method)
t := t0, y0 := x0
 := (tf -t0)/N
for k = 1 to N
{y1 := y0 +  f(t,y0)
(use Euler’s method to find initial guess for Newton’s method)
repeat (Newton’s method)
Solve the linear system {I - 0.5 J[f(ti+1, .)](y1)}y = -G(t,y0,y1,;)
y1 := y1 + y
until || y || < tol
t := t0 +k
y0 := y1}
Output y1, approximation to x(tf)
Newton’s method for two equations
Initialize Define f and its Jacobian F
Set tolerance e,
maximum iteration count -maxits,
starting point (x1, y1)
it := 0.
Loop
Output
Repeat
(x0,y0) := (x1,y1)
it := it + 1
Evaluate f1,f2,F11,F12, F21, F22 at (x0,y0)
D := F11F22 - F21 F12
D1 := f1F22 - f2 F12
D2 := f2F11 - f1 F21
x1 := x0 - D1/D
y1 := y0 - D2/D
until ||(x1,y1) - (x0,y0)|| < e or it = maxits.
If it < maxits then “Solution is (x1,y1)”
else “Failed to converge”.
Iterative method for the solution of ODE
Consider one equation
dy
dx = f(x,y), where
y = y0 and x= x0 are initial points,
x
or y = y0 +
 f ( x, y )dx .
We will solve this equation by iterative
x0
improvement of a solution.
Substituting y0 into the previous equation gives
x
y1(x) = y0 +
 f ( x, A )dx , where A = y0.
x0
For the second approximation of an unknown function it takes the form
x
y2(x) = y0 +
 f ( x, A )dx , where A = y1,
x0
and so forth.
Functions y1(x),..., yn(x) can be considered as the approximation of y(x):
lim yn(x) = y(x)
Exercises.
1. Solve
y =
y +2x/y, y(0) = 1,
using the forward Euler method with  = 0.2 for 0 < t  1
2. Solve
y +
y/x
+y = 0, y(1) = 0.77, y (1) = -0.4
using the forward Euler method with  = 0.1 for 1 < t  1.5
3. Find three sequential approximations for the solution of the equation
y = x2 + y2
where
y(0) = 0.