Download Table 2.5 - 厦门大学数学科学学院

2.3 Newton’s Method
Newton’s (or the Newton-Raphson) method is one of the most powerful and
well-known numerical methods for solving a root-finding problem.
Suppose that f  C 2 [a, b] . Let x  [a, b] be an approximation to p such that
f   x   0 and p  x is “small”. Consider the first Taylor polynomial for f  x 
expanded about x ,
f  x  f  x   (x  x ) f  x  
( x  x )2
f    ,
2
where    x, x  . Since f  p   0 , this equation with x  p gives
0  f  x   ( p  x) f  x  
( p  x )2
f    .
2
Newton’s method is derived by assuming that since
involving
 p x
2
p  x is small, the term
is much smaller, so
0  f  x   ( p  x) f  x  .
Solving for p gives
px
f x
.
f  x 
This sets the stage for Newton’s method, which starts with an initial approximation
p0 and generates the sequence
 pn n 0 , by

pn  pn 1 
f  pn 1 
,
f   pn 1 
for n  1.
Figure 2.7 illustrates how the approximations are obtained using successive tangents.
Starting with the initial approximation
p0 , the approximation
x  intercept of the tangent line to the graph of
f
at
p1
is the
 p , f  p  .
0
0
The
approximation p2 is the x  intercept of the tangent line to the graph of f at
 p , f  p 
1
1
and so on. Algorithm 2.3 follows this procedure.
Algorithm 2.3: To find a solution to f  x   0 given an initial approximation p0 :
1
Input: initial approximation p0 ; tolerance TOL; maximum number of iteration N.
Output: approximate solution p or message of failure.
Step 1 Set i  1
Step 2 While i  N do Steps 3-6
Step 3 Set p  p0 
Step 4
f  p0 
f   p0 
(*Compute pi *)
If p  p0  TOL then
Output(p)
Stop
Step 5 Set
p0  p
Step 6 Set i  i 1
Step 7 Output (The method failed after N iterations)
(* The procedure was unsuccessful *)
Stop.
The stopping-technique inequalities given with the Bisection method are applicable to
Newton’s method. That is, select a tolerance TOL  0 , and construct p1 , p2 , , pi
until
pi  pi 1  TOL ,
pi  pi 1
pi
 TOL, pi  0 , or
f  pi   TOL .
A form of these inequalities can be used in Step 4 of Algorithm 2.3.
This produces the method (Algorithm 2.3) described in the following Subroutine:
SUBROUTINE NEWTONS(N,P0,TOL,P)
C===========================================================
C ALGORITHM 2.3 Newton's Iteration
C PURPOSE
C
To find a solution to P=f(p) given an initial approximation p0
C
INPUT
C
initial approximation: P0
C
tolerance: TOL
C
maximum number of iterations N
C OUTPUT
C
approximation solution P or message of that the method fails
C------------------------------------------------------------------C
INTEGER N
REAL P0,TOL,P
EXTERNAL F,FP
2
INTRINSIC ABS
C *** Print approximation value P using Newtons Iteration ***
9
I=0
WRITE(10,9) I,P0
DO 10 I = 1,N
F0 = F(P0)
FP0 = FP(P0)
D = F0/FP0
P = P0-D
WRITE(10,9) I,P
FORMAT(1X,I5,F12.6)
IF (ABS(P-P0).LT.TOL) THEN
C *** The procedure was successful ***
RETURN
END IF
P0 = P
10 CONTINUE
WRITE(*,*) 'Method failed after N iterations'
RETURN
END
Newton’s method is a functional iteration technique of the form pn  g  pn1  , for
which g  pn 1   pn 1 
f  pn 1 
, for n  1 .
f   pn 1 
In fact, this is the functional iteration technique that was used to give the rapid
convergence was saw in part (e) of Example 3 in Section 2.2.
It is clear from the above equation that Newton’s method cannot be continued if
f   pn1   0 for some n . In fact, we will see that the method is most effective when
f  is bounded away from zero near p .
Example 1: Suppose we would like to approximate a fixed point of g  x   cos  x  .
 
The graph in Figure 2.8 implies that a single fixed-point p lies on 0,  .
 2
C******************************************************************
C
Example 1 (pp68): Using Newton Iteration to find a
C
solution accurate to within 1.0E-6
C
for gives functions f(X) = cosx-x
3
C******************************************************************
C
REAL P0,TOL,P
INTEGER N
OPEN(UNIT=10,FILE='c0.doc',STATUS='UNKNOWN')
PI = 3.1415926
P0 = PI/4.0
TOL = 1.0E-6
N = 50
CALL NEWTONS(N,P0,TOL,P)
WRITE(10,9) P
9
FORMAT(1X,'Approximation solution P:',F12.6)
STOP
END
C
REAL FUNCTION F(X)
C============================================================
C PURPOSE
C
Find the value of function F(X)=cosx-x
C---------------------------------------------------------------------------------------------------C
REAL X
INTRINSIC COS
F = COS(X)-X
RETURN
END
C
REAL FUNCTION FP(X)
C============================================================
C PURPOSE
C
Find the value of function df(X)/dx=-sinx-1
C----------------------------------------------------------------------------------------------------C
REAL X
INTRINSIC SIN
FP = -SIN(X)-1
RETURN
END
Table 2.3 shows the results (accurate to within 106 ) of fixed-point iteration with
p0 

4
. The best we could conclude from these results is that p  0.74 .
4
Table 2.3
-----------------
n
Table 2.4
-------------------
n
pn
----------------0 0.785398
1 0.707107
2 0.760245
3 0.724667
4 0.748720
5 0.732561
6 0.743464
7 0.736128
pn
-------------------0
0.785398
1
0.739536
2
0.739085
3
0.739085
To approach this problem differently, define f  x   cos x  x and apply Newton’s
method. Since f   x    sin x 1 , the sequence is generated by
pn  pn 1 
With p0 
cos  pn 1   pn 1
 sin  pn 1   1
, for n  1.

, the approximations in Table 2.4 are generated. (It notes that if a
4
number is real (in Fortran-77), it has only six effective numerical digit). An excellent
approximation (accurate to within 106 ) is obtained with n  2 . We would expect
this result to be accurate to the places listed because of the agreement of
p2 and
p3 .
The Taylor series derivation of Newton’s method at the beginning of the section
points out the importance of an accurate initial approximation. The crucial assumption
is that the term involving
 p x
2
is, by comparison with p  x , so small that it
can be deleted. This will clearly be false unless x is a good approximation to p . If
p0 is not sufficiently close to the actual root, there is little reason to suspect that
Newton’s method will converge to the root. However, in some instances, even poor
initial approximations will produce convergence.
The following convergence theorem for Newton’s method gives the theoretical
importance of the choice of
p0 。
5
f  C 2  a, b . If
Theorem 2.5: Let
p   a, b is such that
f  p   0 and
f   p   0 , then there exists a   0 such that Newton’s method generates a
sequence
 pn n1

converging to p for any initial approximation .
Proof. The proof is based on analyzing Newton’s method as the functional iteration
scheme pn  g  pn1  , for n  1, with g  x   x 
Let k be in
f  x
.
f  x
 0,1 . We first find an interval  p   , p   
and for which g   x   k , for all
that g maps into itself
x  p   , p   .
Since f  is continuous and f   p   0 , there exists a 1  0 , such that f   x   0
for
x  p  1 , p  1    a, b .
Thus,
is
g
defined
and
continuous
on
 p  1, p  1  . Also,
g  x  1
f   x  f   x   f  x  f   x 
 f   x  
2

f  x  f   x 
 f   x  
2
,
for x  p  1 , p  1  , and, since f  C 2  a, b , we have g  C1  p  1 , p  1  .
By assumption, f  p   0 , so
g p  
f  p  f   p 
 f   p  
2
 0.
Since g  is continuous and , there exits a  , with 0    1 , and
g x  k ,
It remains to show that
for all x  p   , p    .
g
maps
 p  , p  
into
 p  , p  
. If
x  p   , p    , the Mean Value Theorem implies that for some number 
between x and p , g  x   g  p   g    x  p . So
g  x   p  g  x   g  p   g    x  p  k x  p  x  p .
Since x  p   , p    , it follows that x  p   and g  x   p   . Hence, g
6
maps
 p  , p  
 p  , p   .
into
All the hypotheses of the Fixed-Point Theorem are now satisfied, so the sequence
 pn n1 , defined by

pn  g  pn 1   pn 1 
f  pn 1 
,
f   pn 1 
for n  1,
converges to p for any p0  p   , p    .
Theorem 2.5 states that, under reasonable assumptions, Newton’s method converges
provided a sufficiently accurate initial approximation is chosen. It also implies that
the constant k that bounds the derivative of g , and, consequently, indicates the
speed of convergence of the method, decreases to 0 as the procedure continues. This
result is important for the theory of Newton’s method, but it is seldom applied in
practice since it does not tell us how to determine  . In a practical application, an
initial approximation is selected, and successive approximations are generated by
Newton’s method. These will generally either converge quickly to the root, or it will
be clear that convergence is unlikely.
Newton’s method is an extremely powerful technique, but it has a major weakness:
the need to know the value of the derivative of f at each approximation. Frequently,
f   x  is far more difficult and needs more arithmetic operations to calculate than
f  x .
To circumvent the problem of the derivative evaluation in Newton’s method, we
introduce a slight variation. By definition,
f   pn 1   l i m
f  x   f  pn 1 
x  pn1
x  pn 1
.
Letting x  pn  2 , we have
f   pn 1  
f  pn 2   f  pn 1 
pn 2  pn 1

f  pn 1   f  pn 2 
pn 1  pn 2
.
Using this approximation for f   pn1  in Newton’s formula gives
pn  pn 1 
f  pn 1  pn 1  pn  2 
f  pn 1   f  pn  2 
.
This technique is called the Secant method and is presented in Algorithm 2.4. (See
Figure 2.9) Starting with the two initial approximations
7
p0 and
p1 , the
p2 is the x  intercept of the line joining
approximation
 p , f  p  .
1
The approximation
1
 p , f  p 
1
1
and
 p , f  p 
0
0
and
p3 is the x  intercept of the line joining
 p , f  p  , and so on.
2
2
Algorithm 2.4 : Secant method
To find a solution to f  x   0 given initial approximations p0 and p1 ;
Input: initial approximations p0 , p1 ; tolerance TOL;maximum number of iteration N.
Output: approximate solution p or message of failure.
Step 1
i2
q0  f  p0 
q1  f  p1 
Step 2 While i  N do Steps 3-6
Step 3 p  p1  q1  p1  p0   q1  q0 
(*Compute pi *)
Step 4 If p  p1  TOL then
Output  p 
(*The procedure was successful*)
Stop
End if
Step 5 p0  p1
q0  q1
p1  p
q1  f  p
Step 6 i  i 1
Step 7 Output (The method failed after N iterations)
(* The procedure was unsuccessful *)
Stop.
SUBROUTINE SECANT(N,P0,P1,TOL,P)
C================================================
C ALGORITHM 2.4 Secant Method
C PURPOSE
8
C
To find a solution to f(x)=0 given initial approximation p0,p1
C
INPUT
C
initial approximations: P0,P1
C
tolerance: TOL
C
maximum number of iterations N
C OUTPUT
C
approximation solution P or message of that the method fails
C--------------------------------------------------------------------------------C
INTEGER N
REAL P0,P1,TOL,P,Q0,Q1
EXTERNAL F
INTRINSIC ABS
C
DO 10 I = 2,N
Q0=F(P0)
Q1=F(P1)
C*** COMPUTE P(I) ***
P=P1-Q1*(P1-P0)/(Q1-Q0)
WRITE(10,5) I,P
5
FORMAT(1X,I3,2X,F15.7)
IF( ABS(P-P1).LT.TOL ) THEN
RETURN
END IF
C*** UPDATE P0, Q0, P1, Q1
P0=P1
Q0=Q1
P1=P
Q1=F(P)
10
CONTINUE
C
PROCEDURE COMPLETED UNSUCCESSFULLY
RETURN
END
The next example involves a problem considered in Example 1, where we used
Newton’s method with p0   4 .
Example 2. Use the Secant method to find a solution to x  cos x accurate to within
1.0E-6.
Let two initial approximations p0  0.5, p1   4 .
9
Using Secant Method and the formula:
pn  pn 1 
f  pn 1  pn 1  pn  2 
f  pn 1   f  pn  2 
 pn 1 
(cos pn 1  pn 1 )  pn 1  pn  2 
(cos pn 1  pn 1 )  (cos pn  2  pn  2 )
The procedure is detailed in following program
C*******************************************************************
C
Example 1 (pp68): Using Secant Method to find a
C
solution to x=cosx ,accurate to within 1.0E-6
C
with p0=0.5, p1=3.1415926/4
C*******************************************************************
C
INTEGER N
OPEN(UNIT=10,FILE='c0.doc',STATUS='UNKNOWN')
9
PI = 3.1415926
P0 = 0.5
P1= PI/4.0
TOL = 1.0E-6
N = 50
CALL SECANT(N,P0,P1,TOL,P)
WRITE(10,9) P
FORMAT(1X,'Approximation solution P:',F15.7)
STOP
END
C
REAL FUNCTION F(X)
C=============================================
C PURPOSE
C
Find the value of function F(X)=cosx-x
C-------------------------------------------------------------------------C
REAL X
INTRINSIC COS
F = COS(X)-X
RETURN
END
C
Table 2.5 shows the results of the Secant Method applied to x  cos x .
10
Table 2.5
--------------------------------------------n
Pn
--------------------------------------------0
0.5000000
1
0.7853981
2
0.7363842
3
0.7390581
4
0.7390851
5
0.7390851
--------------------------------------------Approximation solution
p:
0.7390851
From Table 2.5, it can be seen that the Secant method is convergence and the
approximation solution p is 0.7390851, which is accurate to the seventh decimal
place.
By comparing the results here with those in Example 1, we see that p5 is accurate to
the seventh decimal place. The convergence of the Secant method is much faster than
functional iteration but slightly slower than Newton’s method, which obtained this
degree of accuracy with p2 .
Newton’s method or the Secant method is often used to refine an answer obtained by
another technique, such as the Bisection method, since these methods require a good
first approximation, but generally give rapid convergence
Homework 3:
Exercise set 2.3
Q5. Use Newton’s method to find solutions accurate to within 106 for the for the
following problems:
(b) x  3x  1  0, [-3,-2] ;
3
2
(d) x  0.8  0.2sin  x   0, [0,  2] .
Q13. The fourth-degree polynomial
f  x   230x4  18x3  9 x2  221x  9
has two real zeros, one in [-1,0] and the other in [0,1]. Attempt to approximate these
zeros to within 106 using the
11
(b) Secant method
(c) Newton’s method
Use the endpoints of each interval as the initial approximations in (b) and the
midpoints as the initial approximation in (c).
12