* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Lecture: 9
Survey
Document related concepts
Mathematical optimization wikipedia , lookup
Simulated annealing wikipedia , lookup
Multi-objective optimization wikipedia , lookup
Finite element method wikipedia , lookup
Quartic function wikipedia , lookup
System of polynomial equations wikipedia , lookup
Multidisciplinary design optimization wikipedia , lookup
Weber problem wikipedia , lookup
Horner's method wikipedia , lookup
System of linear equations wikipedia , lookup
Interval finite element wikipedia , lookup
Newton's method wikipedia , lookup
Root-finding algorithm wikipedia , lookup
Transcript
Lecture: 9 BASES of NUMERICAL SOLUTION of ORDINARY DIFFERENTIAL EQUATIONS (ODE) The lecture discusses theoretical problems of solving ODE and covers Eulers, RungeKutta and predictor-corrector methods. Also the lecture presents an introduction to stiff equation's problem. INPUT PROBLEM and EXAMPLES 1. Statement of the problem Consider du( t ) Cauchy problem for the system of ODE dt = f(t,u), t> 0, u(0) = u(0) (1) or more explicitly dui( t ) dt = fi(t,u1, u2,..., um ), t > 0, i=1, 2, ..., m,. (2) ui(0) = ui(0), i=1,2, ..., m. (3) Conditions guaranteeing the existence and uniqueness of the solution of Cauchy problem are well known. Suppose that functions fi , i=1,2, ..., m, are continues with respect to all arguments on the closed domain D = {|t| a, |ui - ui(0)| b, i=1,2,...m}. From continuity of functions fi it follows their boundedness, that is, existence of the constant M > 0 such that inequalities | fi | M, i=1,2 ,..., m are valid everywhere on D. Suppose, in addition that functions fi satisfy Lipschitz condition with respect to all arguments u1, u2,..., um , e.g. , |fi(t,u’1, u’2,..., u’m ) - fi(t,u’’1, u’’2,..., u’’m )| L{ |u’1 - u’’1| + |u’2 - u’’2| + ... |u’m - u’’m| } for any points t,u’1, u’2,..., u’m and t,u’’1, u’’2,..., u’’m in the domain D. If formulated above conditions are fulfilled then there is a unique solution of the system (2) u1= u1(t), u2= u2(t), ..., um= um(t). This solution is defined for |t| t0 = min(a, b/M) and takes the given initial value (3) for t = 0. So we suppose the solution exists, is unique and has needed smoothness. 2. Examples of numerical methods There are two groups of solutions : multystep difference methods and Runge-Kutta methods. For simplicity consider one equation du( t ) dt = f(t,u), t > 0, u(0) = u (4) 0. Here, it is assumed that we are seeking evenly spaced approximations on some interval. Consider a set of points = {tn = n , n= 0,1,2,...}, where > 0 is a uniform step. Let us define u(t) as an exact solution of the equation (4), and yn = y(tn) as an approximate solution. Note that the approximate solution is a grid function, that is this solution is defined only in the grid nodes. Example 1. Euler’s method. Equation (4) can be considered as a difference equation (yn + 1 - yn) / - f(tn,yn) = 0, n=0,1,2,.., y0 = u0. (5) When we deal with the numerical solution the main question is the convergence of a method. Let us fix a point t . We have a sequence of grids so that 0 and tn = n = t can be created. A method is convergent if |yn + 1 - u(tn)| 0 for 0 and tn = t . As it is usual in numerical analysis, the main question is how to measure the accuracy and efficiency of an approximation scheme. One very useful characterization of a method is its order. We will say that a method is order p if there is a constant C , independent of the stepsize such that |yn + 1 - u(tn)| < C p. We have error zn = yn -u(tn). Since from the equation (5) (zn+1 - zn)/ = f(tn,un + zn) - (un+1 - un)/ we can write (6) the right part of the equation (6) can be represented as the sum (1)n + (2)n, where (1)n = - (un+1 - un)/ + f(tn,un) , (2)n = f(tn,un + zn) -f(tn,un) . The function (1)n is a residual function or an approximation error. Generally, the difference method approximates the initial differential equation if (1)n 0 for 0. The difference method has p-order approximation, if (1)n < Cp. The function (2)n vanishes, if the right part does not depend on the solution u, and in a common case (2)n is proportional to zn. Heuristic confirmation that Euler’s method is order one is provided by Taylor’s theorem. We have for any tn (un+1 - un)/ = u’(tn) + O(), then since the equation (4) (1)n =- u’(tn) + f(tn,un) + O() = O(), Euler’s method is order one. Example 1.1. Solve y = -20y +7 exp(-0.5), y(0) = 5, using the forward Euler method with = 0.01 for 0 < t 0.1 Solution. The first few steps of calculations are shown next: t0 = 0, y0 = y(0) = 5 t1 =0.01 y1 = y0 + y0= 5 +(0.01)(-20(5) + 7 exp (0)) = 4.07 t1 =0.02 y2 = y1 + y1= 4.07 + (0.01)(-20(4.07) + 7 exp (-0.005)) = = 3.32565 The numerical methods to solve first-order differential equations can be easily applied to higher-order ODEs because higher-order ODES can be decomposed to a set of first-order differential equations. For example , consider y + a y + b y + c y + ey = g where a,b,c,d,e, and g are constants. The initial conditions are given as y(0) = y0, y(0) = y0, y(0) = y0, y(0) = y0 By defining u,v, and w as u = y, v = y, w = y it is equivalent to the set of four-order ordinary differential equations: y = u, y(0) = y0 u = v, u(0) = y0 v = w, v(0) = y0, w = g - aw - bv - cu - ey, w(0) = y0 Example 2. Symmetric scheme. The equation (4) is replaced by the difference equation (yn+1 - yn)/i - (f(tn,yn) + f(tn+1,yn+1))/2 = 0, .n = 0,1,..., y0 = u0. (7) This method is an implicit method because the new value y n+1 is defined from the solution of the following equation yn+1 - 0.5 f(tn+1,yn+1) = Fn, where Fn = yn + 0.5 f(tn,yn). An advantage of this method is confined in more high order of approximation accuracy. For the residual function (1)n = -(un+1 - un)/ + (f(tn,yn) + f(tn+1,yn+1))/2 we have the expansion (1)n =- u’n - u’’n/2 + O(2) + (u’n + u’n+1)/2 = (u’n + u’n+1+ - u’n - u’’n/2 + u’’n +O(2))/2, that is (1)n = O(2). Thus the method is order two. Example 3. Runge-Kutta method Suppose we know an approximated solution yn of initial problem for t = tn. To find yn+1 = y(tn+1) at the beginning Euler’s scheme (yn + 1/2 - yn) / 0.5 = f(tn,yn) (8) for calculating the intermediate value yn + 1/2 can be used. Then the difference equation (yn + 1/2 - yn) / = f(tn +0.5 , yn+1/2) (9) is used to define yn+1 explicitly. To investigate the residual (1)n we substitute the intermediate value yn + 1/2 = yn + 0.5 fn into the equation (9). We obtain the difference equation (yn + 1 - yn) / - f(tn +0.5 , yn+0.5fn)=0 for which the (10) residual (1)n is equal (1)n = -(un+1-un)/+f(tn+0.5,un+0.5f(tn,un)), (11) We derive (un+1-un)/=un+0.5u’’n+O(2), f(tn+0.5,un+0.5f(tn,un))= =f(tn,un)+0.5(f(tn,un)/t+0.5f(tn,un)f(tn,un)/u)= =f(tn,un)+ 0.5u’’n, since from the equation (4) we have d2u/dt2=f/ t+f f/u. Thus the method (10) has the order two of error approximation and this method is explicit. This two-step scheme(8) and (9) is known as a predictor-corrector scheme. At the first step (8) we have prediction with the accuracy O(), and at the second step we correct the predicted value. The method (10) can be realized in another way. At first we calculate functions k1 = f(tn,un), k2 = f(tn+ 0.5,yn + 0.5k1), and then define yn+1 from the equation (yn-1 - yn)/ = k2. Such form of realization is called Runge-Kutta method and requires two intermediate calculations of k1 and k2 . STABILITY and STIFF DIFFERENTIAL EQUATIONS In many applications the solution to the initial value problem x = f(t,x), x(t0) = x0 (1) approaches an equilibrium point x = x-, that is, a point for which f(t,x-) 0 for all t. Approximating solutions by numerical means as they approach an equilibrium point might be thought to be routine, but this is not always the case, as the following example shows. Example 1. The solution x(t) = e-t*t to the initial value problem x = - 2tx , x(0) = 1 (2) approaches the equilibrium point x = 0. Figure 1 shows the Euler method approximation for stepsize of = 0.2. It illustrates an instability that is encountered with many fixed-step numerical methods for solving differential equations. Figure 1. Stability of Euler’s Method To see more about the origin of such instability, let us consider a simpler problem, the application of Euler’s method to the initial value problem x = x , x(0) = x0 , (3) where is constant. For Euler’s method y i + 1 =yi + yi= (1+ )yi Thus y0 =x0 , y1 = (1+ )x0 , y2 = (1+ )2x0 , .......................... (4) yi = (1+ )ix0 . When < 0 , the actual solution x(t) =x0 e t approaches the equilibrium point x = 0. The Euler method approximation yi = (1+ )ix0 also approaches x = 0 only if |1 +| < 1, that is, (-2,0). When 1 + < -1, the Euler method approximation will exhibit growing oscillations about x =0. Figure 2 shows the Euler method approximation to the solutions of x’ = -5x , x(0) = 1 when = 0.25 and when = 0.5. Figure 2. The analysis of the stability of Euler’s method for the initial value problem (3) leads us to expect that for the problem (2), Euler’s method will be stable so long as -2t (-2,0).With = 0.2 this gives t (0,5). More generally, suppose that the solution to the initial value problem (1) approaches the equilibrium point x =0. Then from Taylor’s theorem f(t,x) = f(t,0) + f(t,0)/x + O(|x|2) = f(t,0)/x + O(|x|2) (5) since f(t,0) = 0. The initial value problem x= f(t,0)/x, x(t0) = x0 is referred to as the linearized form of (1). When f(t,0)/ x 0 Euler’s method will be stable when f(t,0)/ x is in the interval of stability (-2,0) for the problem (3). Based upon the above considerations, we make the following definition. Definition. The interval of absolute stability of a numerical method is the interval of in which the approximation yi to the solution x(ti) =x0 e to (3) approaches zero as i . Stability of Runge-Kytta Methods The interval of absolute stability of other Runge-Kytta Methods can be found in a similar manner to that used for Euler’s method. Example 2. Let us find the interval of absolute stability for the modified Euler method yi + 1 = yi +0.5 [f(ti,yi) + f(ti + ,yi + f(ti,yi))] . With f(ti,yi) = yi, we have y i + 1 =( 1 + +0.52 2)yi whence y i + 1 =( 1 + +0.52 2)x0. Thus the approximations approach x = 0 when | 1 + Since 1 + +0.522 | < 1. +0.522 = 0.5( +1)2 +0.5, this implies that the interval of absolute stability for the modified Euler method is (-2,0), as it was for Euler’s method. We call a method absolutely stable if it is stable for any . From the general formula for Runge-Kutta method, it is clear that application of the method to the problem (3) will lead to an equation of the form y i+1 =P(-)yi, where - = . For the Euler and modified Euler methods we have found P(-) = 1 + -and P(-) = 1 + -+0.5-2 , respectively. Since y i+1 =P(-)yi implies y i+1 =P(-)ix0 , the condition for absolute stability is |P(-)| < 1. Stiff Systems of Differential Equations To see the implementations of the above analysis let us consider the problem of approximating the solution to the initial value problem x + (b +1) x’ + bx = 0, x(0) = 1, x(0) =0, where b > 0. The solution to this problem is easily seen to be x(t) =[b/(b-1)]e-t - [1/(b-1)]e-bt. Problem (6) is equivalent to the initial value problem (6) x 1 = x2 , x2 = -bx1 - (b + 1)x2, x 1(0) = 1 , x2(0) = 0 (7) for first-order systems. The solution to (7) is x1(t) = [b/(b-1)]e-t - [1/(b-1)]e-bt, x2(t) = -[b/(b-1)]e-t + [b/(b-1)]e-bt. (8) Now suppose that b is much large than 1. Table 1 shows the interval of absolute stability for Runge-Kutta methods with global truncation errors of order one through four. Order P(-) Interval ---------------------------------------------------------------------1 1 + - (-2,0) 2 1 + -+0.5 -2 (-2,0) 3 1 + -+0.5 -2 + 1/6 -3 (-2.51,0) 4 1 + -+0.5 -2 + 1/6 -3 + 1/24 -4 (-2.78,0) Table 1. Intervals of absolute stability for Runge-Kutta methods Then from Table 1 a fourth-order Runge-Kutta method is stable for stepsizes for which - < 2.78/b. If b =1000, for instance, then a stepsize of - < 0.00278 is required. This is far smaller than is required to keep the local truncation error of the method within all but the most stringent tolerances. Indeed, an implementation of the classical Runge-Kutta method in Turbo Pascal with a stepsize of - = 0.0025 produces approximations to the solution (8) that are about the size of the machine unit = 2 10-12. However, if the stepsize is increased to - = 0.003 , then the method is unstable and floating-point overflow quickly results. A system of differential equations such as (6) with large b, in which the stepsize is constrained by the interval of absolute stability rather than the need to keep the truncation error small, is said to be stiff. Methods of numerical approximation such as the Runge-Kutta-Felberg and Gragg methods that efficiently control the local truncation error tend to do poorly on stiff problems also. Obviously the notation of stiffness is very imprecise, depending as it does upon the interplay of the truncation error and the interval of absolute stability of a numerical method. About all that can be said generally is that it arises when the time intervals over which different “components” of a solution undergo changes of comparable magnitude are very different. For instance, in the solution (8) one component decays as e-t while the other decays as e-tb, which will be much faster if b is large. A nonhomogeneous linear system x’(t) = Ax +f(t) could stiff either because the eigenvalues of A differ considerably from each other or because the transient solution decays on a time scale very different from that appropriate for measuring changes in the steady state solution. Approximation of Solutions to Stiff Differential Equations A good approach to solving stiff equations is to use a method that is absolutely stable and thus whose interval of stability cannot become a constraint. The implicit trapezoid rule is such a method. We can interpolate the points (ti,x(ti)) and (ti+1,x(ti+1)) rather then (ti-1,x(ti-1)) and (ti,x(ti)). Then the method of interpolation yi+1 =yi + 0.5 [f(ti,yi +f(ti+1,yi+1)] is the implicit trapezoid method. Applying the implicit trapezoid rule to the initial value problem x = f(t,x), x(t0) = x0 (9) gives G(ti,yi,yi+1;) = yi+1 - yi - 0.5 [f(ti,yi) + f(ti+1,yi+1)] = 0 (10) For known ti and yi this equation must be solved for the approximation yi+1 to x(ti+1). It turns out that for stiff systems simple iteration will not always converge, and thus Newton’s method for systems must be used. Recall that the Newton iteration for solving the system F(x) = 0 is xi+1 = xi -{J[F](xi)}-1F(xi) , where J[F](x) is the Jacobin matrix of F, |F1(x)/x1......F1(x)/xn| J[F](x):= |.........................................| (11) |Fn(x)/x1......Fn(x)/xn| The Jacobean matrix of the function G(ti,yi,yi+1;) is easily seen to be J[G](yi+1)= I - 0.5 J[f(ti + ,.)](yi+1). Mentioned below algorithm outlines a procedure for approximating x(tf) using N steps of implicit trapezoid rule. Algorithm. Fixed-step implicit trapezoid method Input t0,x0 (initial conditions) tf (point at which to find approximation) N (number of subintervals) tol( error tolerance for Newton’s method) t := t0, y0 := x0 := (tf -t0)/N for k = 1 to N {y1 := y0 + f(t,y0) (use Euler’s method to find initial guess for Newton’s method) repeat (Newton’s method) Solve the linear system {I - 0.5 J[f(ti+1, .)](y1)}y = -G(t,y0,y1,;) y1 := y1 + y until || y || < tol t := t0 +k y0 := y1} Output y1, approximation to x(tf) Newton’s method for two equations Initialize Define f and its Jacobian F Set tolerance e, maximum iteration count -maxits, starting point (x1, y1) it := 0. Loop Output Repeat (x0,y0) := (x1,y1) it := it + 1 Evaluate f1,f2,F11,F12, F21, F22 at (x0,y0) D := F11F22 - F21 F12 D1 := f1F22 - f2 F12 D2 := f2F11 - f1 F21 x1 := x0 - D1/D y1 := y0 - D2/D until ||(x1,y1) - (x0,y0)|| < e or it = maxits. If it < maxits then “Solution is (x1,y1)” else “Failed to converge”. Iterative method for the solution of ODE Consider one equation dy dx = f(x,y), where y = y0 and x= x0 are initial points, x or y = y0 + f ( x, y )dx . We will solve this equation by iterative x0 improvement of a solution. Substituting y0 into the previous equation gives x y1(x) = y0 + f ( x, A )dx , where A = y0. x0 For the second approximation of an unknown function it takes the form x y2(x) = y0 + f ( x, A )dx , where A = y1, x0 and so forth. Functions y1(x),..., yn(x) can be considered as the approximation of y(x): lim yn(x) = y(x) Exercises. 1. Solve y = y +2x/y, y(0) = 1, using the forward Euler method with = 0.2 for 0 < t 1 2. Solve y + y/x +y = 0, y(1) = 0.77, y (1) = -0.4 using the forward Euler method with = 0.1 for 1 < t 1.5 3. Find three sequential approximations for the solution of the equation y = x2 + y2 where y(0) = 0.