Download Numerical Methods-module-1-1-1-2

CHAPTER 2
Solution of Non-linear Equations
2.1
INTRODUCTION
We have seen that expression of the form f(x) = a0xn + a1xn – 1 + .... + an –1x + an
where a’s are constant (a0 ≠ 0) and n is a positive integer, is called a polynomial in x of degree
n, and the equation f (x) = 0 is called an algebraic equation of degree n. If f (x) contains some other
functions like exponential, trigonometric, logarithmic etc., then f (x) = 0 is called a transcendental
equation. For example, x3 – 3x + 6 = 0, x5 – 7x4 + 3x2 + 36x – 7 = 0
are algebraic equations of third and fifth degree, whereas x2 – 3 cos x + 1 = 0,
xex – 2 = 0,
x log10 x = 1.2 etc., are transcendental equations. In both the cases, if the coefficients are pure
numbers, they are called numerical equations.
In this chapter, we shall describe some numerical methods for the solution of f(x) = 0 where f(x) is algebraic or
transcendental or both.
 Method for finding the root of an equation can be classified into following two parts:
(a) Direct methods
(b) Iterative methods
Direct Methods:- In some cases, roots can be found by using direct analytical methods. For example, for a
quadratic equation ax2 + bx + c = 0, the roots of the equation, obtained by
√
These are called closed form solution. Similar formulae are also available for cubic and biquadratic polynomial
equations but we rarely remember them. For higher order polynomial equations and non-polynomial equations, it
is difficult and in many cases impossible, to get
Iterative Methods:-These methods, also known as trial and error methods, are based on the idea of successive
approximations, i.e., starting with one or more initial approximations to the value of the root, we obtain the
sequence of approximations by repeating a fixed sequence of steps over and over again till we get the solution
with reasonable accuracy. These methods generally give only one root at a time.
For the human problem solver, these methods are very cumbersome and time consuming, but on other hand,
more natural for use on computers, due to the following reasons:
(1) These methods can be concisely expressed as computational algorithms.
(2) It is possible to formulate algorithms which can handle class of similar problems. For example, algorithms to
solve polynomial equations of degree n may be written.
1
(3) Rounding errors are negligible as compared to methods based on closed form solutions.
 Since, we cannot perform infinite number of iterations; we need a criterion to stop the iterations. We use
one or both of the following criterion:
(i) The equation ( )
| (
is satisfied to a given accuracy or (
) is bounded by an error tolerance ε.
)|
(ii) The magnitude of the difference between two successive iterates is smaller than a given accuracy or an error
|
bound ε. |
For example, if we require two decimal places accuracy, then we iterate until |
|
require three decimal places accuracy, then we iterate until |
|
. If we
.
Common Iterative numerical methods can be classified as:
Bracketing methods: These methods start with guesses that bracket (contain) the root and then systematically
reduce the width of the bracket.
i)
Bisection
ii) False position
Open methods: These methods also involve systematic trial-and-error iterations but do not require that the
initial guesses contain the root.
i)
Secant method
ii) Fixed point iteration
iii) Newton-Raphson method
Theorem: - (Intermediate value theorem)
Let f(x) be continuous in [a, b] and let p be any number between f(a) and f(b), then their exist a number in (a,
b) such that f (c )  p.
Corollary: - If f(x) is continuous in [a, b] and f (a ) f (b)  0 , then f (c)  0 for at least one number c such that
a  c  b.
i)
Bisection Method: The bisection method is a root-finding algorithm which repeatedly bisects an
interval then selects a subinterval in which a root must lie for further processing.
Suppose ( ) is continuous on an interval [a, b] such that
Then the first estimate of the root,
is given by
( ) ( )
,
,
We make the following evaluations to determine the subinterval in which the root lies:
If
( )
then
is the root of . Otherwise
i)
if
( ) ( )
then the root lies in the subinterval ,
-
ii)
If
( ) ( )
then the root lies in the subinterval ,
-
Continue the process until either we get the exact root or we may have an approximate root with the required
degree of accuracy.
2
The Bisection method generates a sequence *
i.e
(
)
+ of mid points of the reduced interval.
where
Condition of convergence Theorem: Assume that
,
exists a number
is,
- such that
( )
is continuous on ,
. The sequence *
- and ( ) ( )
+ converges to the zero
, then there
. That
.
Example 1: Approximate the root of
( )
( )
& also
( )
in [2, 3].
is cont on [2, 3]. Thus
has at least one root b/n 2 & 3.
(
)
1
2
3
2.5
5.6250 > 0
2
2
2.5
2.25
1.8906 > 0
3
2
2.25
2.125
0.3457 > 0
4
2
2.125
2.0625
-0.3513 < 0
5
2.0625
2.125
2.09375
-0.0089 < 0
The desired approximation is
if |
|
|
Theorem: Termination criteria for a given tolerance (acceptance) or error
|
:
The number of iteration ( ) process will be terminated when the length of the interval (
small. i.e
3
) becomes very
Remark: - In order that the absolute error mn   
ba
2n
 ba
ln
 
required to achieve this accuracy is given by n 
ln 2
  (a given tolerance), the number of iterations



 =
0
(
)
1
Example 2: Determine approximately how much iteration is necessary to solve
( )
, with an accuracy of
Solution: The maximum number of iteration
(
It requires
for
and
is given by
(
)
)
iterations to obtain an approximation to an accurate of
3
Example3:- Find the number of iterations needed to approximate the positive root of x  2 x  5  0 with an
absolute error < 0.001 and also find that approximate root.
( )
Solution:
Tolerance,
( )
.
.,
/
The number of iterations needed to achieve the tolerance 0.001 is an integer greater than or equals 9.965784, i.e.
at least for n = 10.
( )
( )
( )
( )
( )
( )
( )
( )
( )
(
)
See that │
Example4: - Use Bisection Method to approximate the root of x3 –x2 -1 = 0 correct to three decimal places.
(Proceed until the first three decimal places of two consecutive iterates agree.)
Solution:- The following Fig. Matlab graph of x3 –x2 -1 on [-3, 3]
4
20
10
0
-10
-20
-30
-40
-3
( )
-2
-1
0
1
2
3
( )
( )
( )
( )
( )
( )
( )
( )
( )
( )
(
)
(
)
(
)
(
)
(
)
Therefore, the root correct to three decimal places is 1.465
Example 5: Using the bisection method, find the real root of the equation ( )
Sol.
The given equation at
, ( )
√
. Therefore, a root lies between 0 and 1
and at
5
√
, ( )
√
then (
First approximation:
root lie b/n
)
(
)
(
√
)
. This shows the
.
2nd appx.
then (
3rd appx.
)
then (
lies b/n
(
)
)
(
√
(
)
)
. Thus the root lies b/n
(
√
)
. Thus the root
.
4th appx.
then (
root lies b/n
)
(
)
(
√
)
. Thus the
.
5th appx.
the root lies b/n
7th approximation:
)
(
)
√
(
)
. Thus
then (
)
(
)
√
(
)
. Thus
.
6th appx.
the root lies b/n
then (
.
.
From the last two observations, that is, x6 = 0.3907 and x7 = 0.3985, the
approximate value of the root up to two places of decimal is given by 0.39. Hence the
root is 0.39 approximately.
Example6: - Use the Bisection Method to approximate a) √
b) √
accurate to within
i) 10-2
ii) 10-4
Solution: - i)
3 is one root of the equation f ( x)  0, where f ( x)  x 2  3 and it is located between
1 and 2
Since
01 after 6 iterations the root is:
ii)
6
Since
is: 1.73204041
after 16 iterations the root
The fastness of convergence towards a root in any method is represented by its rate of convergence.
Remarks: - The Bisection Method
 always converge
 Since the error decrease at each step by a factor of ½ convergence in this method has order one (or linear).
 Very reliable and has good error bounds
 It converges slowly.
2. Method of False position /Regula Falsi Method /linear interpolation method: One of the shortcoming of
bisection method is that, in dividing the interval , - no account is taken of the magnitude of ( )
( ) For
example if ( ) is much closer to zero than ( ) then it is likely that the root is closer to than .
An alternative method, false position that exploits this graphical insight is to join ( )
(chord).
( ) by a straight line
The intersection of this line with x-axis represents an improved estimate of the root.
Procedure: In this method we choose two points
of ( )
must lie in between these points, given that
( )
( ) are of opposite signs. Hence, a root
is continuous on ,
Now the equation of the chord joining the two points (
( )) & (
( )
( )
The intersection of this line with the x-axis is the -intercept say
(
(
)
)
(
(
)
Check the functional value of
If ( )
( )
)
(
(
)
)
(
( )
then x3 is the exact root of ( )
7
)
( )) is given by
( )
which is the first estimate of the root;
If
(
( )
the root is in [ , ] (considering ( )
), In this case join the points (
( )) using a chord to get the next better approximate root .
( )) &
Continue the process by bracketing the root until the root is estimated adequately by;
(
)
(
(
)
)
(
)
, for the interval ,
- that contains the root
Condition of convergence Theorem:
,
Assume that
is continuous on [a, b] and there exists a number
opposite signs, and
(
(
)
(
- such that ( )
. If ( )
( ) have
)
)
(
)
represents the sequence of points generated by the false position process, then the sequence
zero
.
That is,
.
converges to the
Termination criteria for a given tolerance of error : Here, we use the size of | ( )| as the stopping criterion for
| ( )|
it.
Example: Find a root of the equation
in [2, 3], using regula falsi method up to 4th iteration.
( )
Solution: The formula for the method is
- ( )
Since ( )
)
(
Now find ( )
)
)
(
)
( )
(

(
(
)
the root lies b/n 2 & 3
=
(
=
)
( )
Now use
(
)
(
(
(
)
)
(
the root is in ,
)(
-
)
)
etc.
Remark: This method
 Is faster than bisection method
 Has a limitation when there is a discontinuity over an interval, when there are distinct roots and over a
"large" interval.
Open Methods:-these methods do not contain the root by an interval.
8
a)The Secant method: The secant method is an open method of root-finding algorithm that uses a succession of
roots of secant lines to better approximate a root of a function f. It resembles totally the method of false position
except that no attempt is made to ensure the root is enclosed. This method needs two initial values (initial
approximate roots).
Procedure:
of
Let
( ) i.e
be the root of
(
, and assume
( ))
. Draw the secant line passing through(
-axis, say
( )
(
be two initial approximate roots
( )). Find the point at which the line crosses the
and from the slope of the line, we have
)
(
)
(
)
- ( )
Further approximations of
– (
)
(
(
(
(
)
)
(
)
are computed using the iteration;
)
)
(
)
Condition of convergence Theorem:
The iterates
of the secant method converge to a root of f, if the initial values and are sufficiently close to the
root. If the initial values are not close to the root, then there is no guarantee that the secant method converges.
Termination criteria within a given tolerance : The stopping criterion for this method is the size of | (
Example1: Use the secant method to approximate the root of
&
& | ( )| < =
( )
Solution:
(
(
),
)
( ),
( )
(
)
( )
,
(
( )
( )
)
. and
,
( )
-
9
)|
with initial approximations
And
( )
–
( ),
( )
and
(
)
( )
and
( )
. . . so on.
accurate to within 10-4.
Example2: Use Secant Method to find a root of the equation
Solution: ( )
Taking
, the root lies between
.
and
x2  x1 
x1  x0
* f ( x1 )
f ( x1 )  f ( x0 )
= 0.61108428
x3  x 2 
x2  x1
* f ( x2 )
f ( x2 )  f ( x1 )
= 0.72328609
x 4  x3 
x3  x 2
* f ( x3 )
f ( x3 )  f ( x 2 )
= 0.73956633
x5  x 4 
x 4  x3
* f ( x4 )
f ( x 4 )  f ( x3 )
= 0.73908347
x 6  x5 
x5  x 4
* f ( x5 )
f ( x5 )  f ( x 4 )
= 0.73908514
x6  x5  .0001 and hence the root to the given accuracy is : 0.73908347
Remark: In this method
We are not sure about the convergence of the approximated roots to the exact root. But, the process is simpler;
because the sign of ( ) is not tested, and often converges faster.
Exercise: Use the method of regula falsi method to find the root
| (
)|
in the interval ,
-
( )
until
(Hint: Use radian measure).
c) Fixed point Iteration
This method is also known as the direct substitution method. To find the root of the equation ( )
successive approximations we rewrite the given equation in the form of
( ).
A number is a fixed point for a given function
equivalent classes in the following sense:
if ( )
10
by
. Root-finding problems and fixed point problems are
Given a root finding problem ( )
, we can define functions with a fixed point at in a number of ways,
( ) or as ( )
for example, as ( )
( ). Conversely, if the function has a fixed point at , then
the function defined by ( )
( ) has a zero at .
To approximate the fixed point of a function , we choose an initial approximation and generate the sequence
* + by letting
(
), for each
. If the sequence converges to
is continuous, then
(
)
(
)
( ) and a solution
( ) is obtained. This technique is
called fixed point iteration or functional iteration. The procedure is illustrated in the figure below.
Example 1 The function ( )
and ( )
for
has fixed points at
There are many ways to change the equation ( )
manipulation.
For example the function ( )
iv)
With
( )
.
)
( ) using simple algebraic
to the fixed point form
can be changed to the fixed point form as
( )
i)
since (
and
( )
ii)
/
v)
.
/
iii)
( )
(
)
( )
, the table below lists the results of the fixed point iterations for all five choices of .
()
( )
(
( )
( )
1.5
( )
)
1.365230013
7
8
9
11
10
15
20
25
30
The actual root is 1.365230013. It can be seen that excellent results have been obtained for choices iii), iv) and
v), It is interesting to note that choice i) was divergent and that ii) became undefined because it involved the
square root of a negative number.
Even though the various functions in the above example are fixed point problems for the same root-finding
problems, they differ vastly as techniques for approximating the solution to the root finding problem.
How can we find a fixed point problem that produces a sequence that reliably and rapidly converges to a
solution to a given root-finding problem?
The following theorem gives us some clues concerning the paths we should pursue and, perhaps more
importantly, some we should reject.
Theorem 2.1. (Fixed point theorem)
- for all
,
-. Suppose in addition that
Let is continuous on ,
( ) ,
(
). Then for any number
that a constant
exists with | ( )|
, for all
,
(
)
defined by
converges to the unique fixed point
Example 2: Discuss the convergence of the method for different choices of
) and
exists on (
,
- the sequence
on example 1 above.
a) For ( )
, we have ( )
and ( )
, so
does no map , - in to itself.
, -. Although theorem 2.1 does not guarantee that
( )
Moreover,
, so | ( )|
for all
the method must fail for this choice of , there is no reason to expect convergence.
b) With
( )
.
defined when
|
( )|
does not map ,
/ , we can see that
- in to ,
. Moreover there is no interval containing
, since |
c) For the function
( )|
( )
- and the sequence *
+
is not
such that
. There is no reason to expect this method will convergence.
(
) ,
( )
(
,
)
-,
So
is strictly decreasing on , -. However,| ( )|
, so the condition | ( )|
fails on , -.
A closer examination of the sequence * +
starting with
shows that it suffice to consider the interval
,
- instead of , -. On this interval it is still true that
( )
is strictly decreasing, but , additionally,
,
- this shows that maps the interval ,
- on to
( )
( )
( )
for all
itself. Since it is true that | ( )| | ( )|
on this interval, theorem 2.1 confirms the convergence of
which we are already aware.
d) For
( )
.
/ , we have |
( )|
|
√
(
)
|
for all
√
12
,
-
( ) is much smaller than the bound (found in (c)) on the magnitude of
The bound of the magnitude of
which explains the more rapid convergence using .
e) The sequence defined by
( )
( )
converges much more rapidly than the other choices.
Example3. Find all zeros of ( )
by using the fixed point iteration method for an appropriate
iteration function . Find the zeros accurate to within
Solution: Using graphical method or IVT it can be shown that ( ) has four roots and changes signs in each of the
- ,
- ,
- ,
-. Therefore each of these intervals contains only one
intervals ,
root.
( ) can be changed in to the fixed point function in many ways. Among them
( ). With the initial approximation
i)
( )
(
)
( )
(
)
( )
(
)
( )
(
)
( )
(
)
( )
(
)
( )
(
)
|
|
|
( )
( )
precede the iteration as follows:-
we
( )
.
|
The first zero of
( ) accurate to within
which is in ,
is
-.
For other zeros their approximation function, initial approximations are summarized in the table below.
Interval
,
initial approximation
accurate to within
-
approximating function
( )
(
)
(
)
,
-
( )
,
-
( )
13
d) Newton - Raphson Method
Newton’s method is one of the most powerful and well known numerical methods for solving a rootfinding problem. There are many ways of introducing Newton’s method. One means of introducing
Newton’s method is based on Taylor Polynomial. In Newton’s method, instead of constructing the chord
between two points on the curve of ( ) against (as done in linear interpolation method), the tangent to the
curve is notionally constructed at each successive value of
which the tangent cuts the axis ( )
If the
(
)
( )
, and the next value,
value is
, the tangent to the curve of ( ) at that point has slope
( )). Its equations is thus
(
(
The value of
scheme
) (
)
(
at which
)
is taken as
(
(
)
)
(
, is taken as the point at
) and passes through the point
------------------------------------- (i)
thus the condition
yields from (i) above , the iteration
-------------------------------------- (ii)
This is Newton’s – Raphson iteration formula.
Remark:
 Note that if any of
comes close to the stationary point of , so that ( ) is close to zero, the scheme is
not going to work well.
 The iteration procedure terminates when the relative error for two successive approximations becomes less
than or equal to the prescribed tolerance.
14
Eample1. Find real root of the equation
b/n
Solution: let ( )
and
, now ( )
by Newton’s –Raphson method
and
. Therefore the root lies in ,
( )
-
Newton’s- Raphson method becomes
(
(
)
)
Let
1st appx;
3rd appx:
Sinec
(
)
(
(
2nd appx;
( )
(
4th appx:
)
)
(
)
)
(
)
, The root of the equation is 4.5616.
Example 2: Use the Newton’s – Raphson method to estimate the root of ( )
of
employing an initial guess
( )
Solution. The first derivative of the function can be evaluated as
This can be substituted along with the original function in to (ii).
Starting with an initial guess of
, this iterative equation can be applied to compute
| |
0.0000220
Thus, the approach rapidly converges on the true root.
Exercises
1. Use the Bisection method to find a solution accurate to with in
i)
for the following problems
ii)
2. Let ( )
. With
a) Use the secant method
and
, find
b) Use the method of false position
3. Using the a) secant method b) Method of false position c) Newton’s Method to find the solution accurate to with
in
,
for the following problems i)
15
-
ii)
0
1
4. Using the a) secant method b) Method of false position c) Newton’s Method to find the solution accurate to with
in
for the following problems i)
ii)
5. Employ fixed point iteration to locate the root of ( )
until
(√ )
6. Find √ using Newton – Raphson method taking
use an initial guess of
and iterate
.
7. Use a fixed point iteration method to determine a solution accurate to with in
for
on ,
-.
Use
8. Use a fixed point iteration method to determine a solution accurate to with in
( )
9. Determine the lowest positive root of ( )
a) Using the secant method (three iterations,
b)Using the Newton – Raphson method (three iterations
16
)
for
,
-
CHAPTER THREE
Solutions of System of equations
A system of algebraic equations has the form
(3.1)
where the coefficients
and the constants
notation the equations are written as
are known, and
represent the unknowns. In matrix
(3.2)
or, simply
(3.3)
There are two classes of methods for solving systems of linear, algebraic equations: direct and iterative
methods. The common characteristic of direct methods is that they transform the original equations
into equivalent equations (equations that have the same solution) that can be solved more easily
Iterative, or indirect methods, start with a guess of the solution x, and then repeatedly refine the
solution until a certain convergence criterion is reached. Iterative methods are generally less efficient
than their direct counterparts due to the large number of iterations required. But they do have
significant computational advantages if the coefficient matrix is very large and sparsely populated (most
coefficients are zero).
3.1 Direct methods for solving system of linear equations
From the direct methods we have studied Gauss elimination, Gauss-Jordan elimination and matrix
inversion methods in our previous courses. Here, form the direct methods we concentrate on matrix
decomposition methods
LU Decomposition Methods
It is possible to show that any square matrix A can be expressed as a product of a lower triangular matrix
L and an upper triangular matrix U:
(3.4)
1|Page
The process of computing and for a given A is known as
decomposition or
factorization.
decomposition is not unique (the combinations of and for a prescribed are endless), unless certain
constraints are placed on or . These constraints distinguish one type of decomposition from another.
Three commonly used decompositions are listed in Table 3.2.
Name
Constraints
Doolittle’s decomposition
= 1, i =1,2,...,n
Crout’s decomposition
=1, i =1,2,...,n
Choleski’s decomposition
Table 3.1
After decomposing A, it is easy to solve the equations
. We first rewrite the equations as
. Upon using the notation
, the equations become
which can be solved for y by forward substitution. Then
will yield x by the back substitution
process. The advantage of
decomposition over the Gauss elimination method is that once is
decomposed, we can solve
for as many constant vectors b as we please. The cost of each
additional solution is relatively small, since the forward and back substitution operations are much less
time consuming than the decomposition process.
3.1.1 Doolittle’s Decomposition Method
Decomposition Phase
Doolittle’s decomposition is closely related to Gauss elimination. In order to illustrate
the relationship, consider a 3×3 matrix A and assume that there exist triangular matrices
such that A = LU. After completing the multiplication on the right hand side, we get
Let us now apply Gauss elimination to Eq. (above). The first pass of the elimination procedure consists of
choosing the first row as the pivot row and applying the elementary operations
2|Page
, The result is
In the next pass we take the second row as the pivot row, and utilize the operation
ending up with
The foregoing illustration reveals two important features of Doolittle’s decomposition:
o
o
The matrix U is identical to the upper triangular matrix that results from Gauss elimination.
The off-diagonal elements of L are the pivot equation multipliers used during Gauss elimination;
that is,
is the multiplier that eliminated
.
It is usual practice to store the multipliers in the lower triangular portion of the coefficient matrix,
replacing the coefficients as they are eliminated ( replacing
). The diagonal elements of L do not
have to be stored, since it is understood that each of them is unity. The final form of the coefficient
matrix would thus be the following mixture of L and U:
Solution Phase: Consider now the procedure for solving
of the equations is (recall that
)
Solving the
equation for
by forward substitution. The scalar form
yields
(3.5)
3|Page
The back substitution phase for solving
is identical to that used in the Gauss elimination method.
EXAMPLE 3.1
Use Doolittle’s decomposition method to solve the equations
,where
Solution: We first decompose A by Gauss elimination. The first pass consists of the elementary
operations
Storing the multipliers
and
in place of the eliminated terms, we obtain
The second pass of Gauss elimination uses the operation
Storing the multiplier
in place of
, we get
The decomposition is now complete, with
Solution of
equations is
4|Page
by forward substitution comes next.The augmented coefficient form of the
The solution is
Finally, the equations
, or
are solved by back substitution. This yields
3.1.2 Choleski’s Decomposition
Choleski’s decomposition
o
o
has two limitations:
Since
is always a symmetric matrix, Choleski’s decomposition requires A to be symmetric.
The decomposition process involves taking square roots of certain combinations of the elements
of A. It can be shown that in order to avoid square roots of negative numbers A must be positive
definite.
Although the number of multiplications in all the decomposition methods is about the same, Choleski’s
decomposition is not a particularly popular means of solving simultaneous equations due to the
restrictions listed above. We study it here because it is invaluable in certain applications.
Let us start by looking at Choleski’s decomposition
(3.6)
of a 3×3 matrix:
After completing the matrix multiplication on the right hand side, we get
5|Page
(3.7)
Note that the right-hand-side matrix is symmetric, as pointed out before. Equating the matrices and
element-by-element, we obtain six equations (due to symmetry only lower or upper triangular
elements have to be considered) in the six unknown components of L. By solving these equations in a
certain order, it is possible to have only one unknown in each equation.
Consider the lower triangular portion of each matrix in Eq. (3.7) (the upper triangular portion would do
as well). By equating the elements in the first column, starting with the first row and proceeding
downward, we can compute
,
and
in that order:
The second column, starting with second row, yields
Finally the third column, third row gives us
We can now extrapolate the results for an
triangular portion of
is of the form
and
:
:
matrix. We observe that a typical element in the lower-
Equating this term to the corresponding element of
yields
(3.8)
The range of indices shown limits the elements to the lower triangular part. For the first column (j =1),
we obtain from Eq. (3.8)
(3.9)
6|Page
Proceeding to other columns, we observe that the unknown in Eq. (3.8) is
(the other elements of L
appearing in the equation have already been computed). Taking the term containing
outside the
summation in Eq. (3.8), we obtain
If
(a diagonal term), the solution is
(3.10)
For a non-diagonal term we get
(3.11)
EXAMPLE 3.2
Compute Choleski’s decomposition of the matrix
Solution: First we note that A is symmetric. Therefore, Choleski’s decomposition is applicable, provided
that the matrix is also positive definite. An a priori test for positive definiteness is not needed, since the
decomposition algorithm contains its own test: if the square root of a negative number is encountered,
the matrix is not positive definite and the decomposition fails. Substituting the given matrix for A in Eq.
(3.7), we obtain
Equating the elements in the lower (or upper) triangular portions yields
7|Page
Therefore,
The result can easily be verified by performing the multiplication
.
3.2 Iterative Methods
So far, we have discussed only direct methods of solution. The common characteristic of these methods
is that they compute the solution with a finite number of operations. Moreover, if the computer were
capable of infinite precision i.e. no round off errors, the Solution would be exact.
Iterative, or indirect methods, start with an initial guess of the solution x and then repeatedly improve
the solution until the change in x becomes negligible. Since the required number of iterations can be
very large, the indirect methods are, in general, slower than their direct counterparts. However,
iterative methods do have the following advantages that make them attractive for certain problems:
o
o
For large systems with a high percentage of 0 entries, these techniques are efficient in terms of
both computer storage and computation.
Iterative procedures are self-correcting, meaning that round off errors(or even arithmetic
mistakes)in one iterative cycle are corrected in subsequent cycles.
A serious drawback of iterative methods is that they do not always converge to the solution. It can be
shown that convergence is guaranteed only if the coefficient matrix is diagonally dominant. The initial
guess for x plays no role in determining whether convergence takes place—if the procedure converges
for one starting vector, it would do so for any starting vector. The initial guess affects only the number of
iterations that are required for convergence.
Diagonal Dominance An
matrix is said to be diagonally dominant if each diagonal element is
larger than the sum of the other elements in the same row(we are talking here about absolute values).
Thus diagonal dominance requires that
8|Page
For example, the matrix
Is not diagonally dominant, but if we rearrange the rows in the following manner
Then we have diagonal dominance.
3.2.1 Gauss Jacobi Method
The Gauss Jacobi iterative method is obtained by solving the
obtain (provided
For each
equation in
for
to
)
, generate the components
of
from the components of
by
(3.12)
Example 3.3: The linear system
given by
has the unique solution
to
starting with
9|Page
. Use Jacobi’s iterative technique to find approximations
until
Solution: We first solve equation
for
, for each
From the initial approximation
Additional iterates,
below.
to obtain
we have
(
given by
) are generated in a similar manner and are presented in Table
We stopped after ten iterations because
In fact
.
3.2.2 The Gauss-Seidel Method
A possible improvement of Gauss Jacobi can be seen by reconsidering Eq. (3.12). The components of
are used to compute all the components
10 | P a g e
of
. But, for
, the components
,...,
of
have already been computed and are expected to be better approximations to the actual solutions
than are
. It seems reasonable, then, to compute
recently calculated values. That is, to use
using these most
(3.13)
for each
instead of Eq. (3.12). This modification is called the Gauss-Seidel iterative
techniqueand is illustrated in the following examples.
Example 3.4
Solve the equations
by the Gauss–Seidel method.
Solution: With the given data, the iteration formulas in Eq. (3.13) become
Choosing the starting values
The second iteration yields
11 | P a g e
, we have for the first iteration
and the third iteration results in
After five more iterations the results would agree with the exact solution
five decimal places.
with in
Example 3.5 Use the Gauss-Seidel iterative technique to find approximate solutions to
starting with
and iterating until
Solution The solution
was approximated by Gauss Jacobi method in Example 3.3. For
the Gauss-Seidel method we write the system, for each
as
12 | P a g e
When
,wehave
give the values in table below
. Subsequent iterations
is accepted as a reasonable approximation to the solution. Note that Gauss Jacobi method in
Example3.3 required twice as many iterations for the same accuracy.
Exercises
13 | P a g e
4.2. Interpolation
( )
The statement
means corresponding to every value of in the range
there exists one or more values of . Assuming that ( ) is single valued and continuous
and that is known explicitly, then the values of ( ) corresponding to certain given values of , say
can easily be computed and tabulated. The central problem of numerical analysis is the
)(
) (
)
(
) satisfying the
converse one; given the set of tabular values (
( ) where the explicit nature of ( ) is not known, it is required to find a simpler function
relation
say ( ), such that ( )
( ) agree at the set of tabulated points. Such a process is called
interpolation. If ( ) is a polynomial, then the process is called polynomial interpolation and ( ) is
called the interpolating polynomial. Polynomial interpolation consists of determining the unique
order polynomial that fits n+1 data points. This polynomial then provides a formula to compute
intermediate values. Although there is one and only one nth order polynomial that fits n+1 points, there
are a variety of mathematical formats in which this polynomial can be expressed.
4.2.1 Newton’s formula for interpolation
4.2.1.1 Newton’s forward difference interpolation
)(
) (
)
(
Given the set of
values, i.e. (
find ( ) a polynomial of the
degree such that the functions
points.
Let the values of
be equidistant, i.e.
This means
step size. Since
( )
( )
; where is the interval of differencing called
degree, it may be written as
( ) is a polynomial of the
(
)
(
)(
)
and substituting
(
)
(
)(
)
(
)
(4.1)
( ) should agree at the set of tabulated points, we obtain
Imposing now the condition that
Setting
) of
it is required to
( ) agree at the tabulated
equation (4.1) gives
(
)(
)
(
)(
) (
(
))
(4.2)
(4.2) is called Newton’s forward interpolation formula and useful for interpolation near the beginning of
a set of tabulated values.
1|Page
Example1. Find the cubic polynomial which takes the following values. Hence, or otherwise obtain ( )
x
0
1
2
3
y
1
0
1
10
Solution Form the difference table
x
0
y
1
1
0
2
1
-1
2
1
3
Here
gives
6
8
9
10
. Hence using the formula
. Substituting this value of
( )
(
and choosing
in (4.2) we get
(
)
)
(
)(
we obtain
this
)
This is a polynomial from which we obtain the above tabular value. To compute y(4) we observe that
, hence formula (4.2)gives
( )
(
)
(
)
(
)(
)
That is the same value as that obtained by substituting
we call extrapolation
in the cubic polynomial above. This is what
4.2.1.2 Newton’s backward difference interpolation
Instead of assuming
( )
(
( ) as in (4.1) if we choose it in the form
)
Imposing the condition that
(4.3) we obtain
(
)(
)
(
)(
)
(
( ) should agree at the tabulated points
and setting
and substituting
2|Page
in equation (4.3) gives
)
(4.3)
in
(
( )
)
(
)(
)
(
)(
) (
(
))
(4.4)
This is Newton’s backward difference formula and it uses tabular values to the left of
therefore useful for interpolation near the end of the tabular values.
. This formula is
Example 2 The population of a town in the decennial census was given below. Estimate the population
for the year 1925
year x
1891
1901
1911
1921 1931
population y (in thousands)
46
66
81
93
101
Solution Here interpolation is desired at the end of the table and so we use formula (4.4) with
with
we obtain
(
)
The backward difference table
x
1891
y
46
1901
66
20
2
15
1911
81
1921
93
12
8
101
1931
Hence using formula (4.4)
(
( )
(
(
)
(
)
)(
(
)(
)(
)(
)(
)
)(
)
(
(
)(
)(
Therefore the population in the year 1925 is
)(
)(
)(
)(
)
)
)
thousands.
Additional examples
3. Find ( ) given that ( )
3|Page
( )
( )
( )
The third difference being constant.
Solution Since ( )
(
)
(
( )
)
( )
( )
( )
( )
( )
( )
Difference table
x
y
( )
0
( )
( )
9
1
6
9
2
2
8
4
3
12
Higher differencing being zero
We have ( )
(
( )
)
( )
( )
(
( )
)
( )
( )
(
) ( )
(
)
( )
4. Construct the backward difference table from the data
, assuming the third difference to
be constant, find the value of
Solution First construct the backward difference table
x
25
y
30
0.07336
35
0.0692
40
45
4|Page
0.0643
(
Therefore,
(
)
(
)
(
(
(
(
)
(
)
)
(
) then
and
)
)
)
(
)
this yields
(
this yields
(
)
and
)
Exercises
1. Evaluate
(
a)
2. Prove that ( )
(
))
( )
b)
( )
3. The table below gives the values of
x
Find i)
0.10
0.1003
ii)
0.15
0.20
0.1511 0.2027
iii)
(
)
( )
( )
for
0.25
0.30
0.2553 0.3093
iv)
4.2.2 Lagrange’s Interpolation formula
The disadvantage of Newton’s interpolation formula is it requires the values of the independent variable
to be equally spaced .The Lagrange’s interpolation formula is used to interpolate with unequally spaced
values of the argument.
), given the
Let ( ) be continuous and differenciable
times in the interval (
points
(
)(
) (
)
(
) where the values of need not necessarily be equally spaced,
we wish to find a polynomial of degree , ( ) such that
( )
( )
(4.5)
a) First consider the Lagrange’s polynomial of degree one passing through two points
(
)
(
)
Let the polynomial be
( ) such that
( )
Where
From (4.7) we see that
5|Page
( )
( )
( )
( )
{
( )
∑
( )
(4.6)
(4.7)
(4.8)
Therefore ∑
( )
( )
( )
(4.9)
b) Lagrange’s polynomial of degree two passing through (
)(
)
(
)
Similarly the quadratic Lagrange’s polynomial is written as
( )
∑
(
( )
)(
(
)(
)
(
)
)(
(
)
)(
(
)
(
)(
)
)(
(4.10)
)
Where ( ) satisfy the condition given in (4.8) and (4.9)
To derive the general formula, let
( )
(4.11)
be the desired polynomial of the
degree such that the condition (4.5)[called the interpolatory
conditions] are satisfied. Substituting these conditions in (4.11)
(4.12)
The set of equation(4.12)has a solution if
|
|
Equation (4.11) is a linear combination of
( )
∑
Where ( ) are polynomials in
Equation (4.9) gives
( )
(
(
( )
(4.13)
( )
of degree . Since
( )
for
. Hence ( ) may be written as
( )
)(
)(
. Hence we write
)
)
(
(
)(
)(
)
(
)
)
(
)
)(
)(
(
which satisfies condition (4.8)
If we now set
6|Page
( )
(
)(
)
(
)
(
)
)
( )
( )
Then
( )
(
( )
)
( )
( )
(
|
)(
)
(
)(
)
(
so that
, therefore (4.13) becomes
( )
∑
(
( )
)
(
)
(
)
This is called Lagrange’s interpolation formula. The coefficients ( ) defined in (4.14) are called
Lagrange’s interpolation coefficient. Interchanging
in (4.15) we obtain the formula
( )
( )
∑
(
( )
)
(4.16) is useful for inverse interpolation. The major advantage of this formula is that the coefficients in
(4.18) are easily determined. Further it is applicable to either equal or unequal intervals and the
abscissae
need not be in order.
Example 5. Certain corresponding values of
are given in the table below. Find
300
304
305
307
2.4771 2.4829 2.4843 2.4871
Solution From formula (4.15) we obtain
(
(
(
(
)(
)(
)(
)(
)(
)(
)(
)(
)
(
)
)
(
)
(
(
(
(
)
)
)(
)(
)(
)(
)(
)(
)(
)(
)
(
)
)
(
)
)
)
Example 6 Find the Lagrange’s interpolating polynomial of degree2 approximating the function
defined by the following table of values. Hence determine the value of
Solution
( )
∑
2
0.69315
( )
(
)(
)(
(
(
(
)(
)(
(
7|Page
)
(
)
2.5
0.91629
( )
)
)(
(
(
)
)
3.0
1.09861
( )
(
)
)(
)(
)(
)(
(
(
( )
)
)
)
(
(
(
)
)(
)
)
)(
)(
)
)
(
(
(
(
)
)
)(
)
)(
(
)
)
. This is the required polynomial
(
)
. The actual value of
So that | |
Example 7 Using Lagrange’s interpolation formula, find the form of the function ( ) form the table
0
-12
1
0
3
12
Solution Since
( )
4
24
, it follows that
is a factor. Let ( )
(
) ( ). Then
and ( ).
. We now tabulate the values of
0
3
4
( ) 12
6
8
Applying Lagrange’s formula to the above table, we find
(
( )
(
(
)(
)(
)(
)
)
(
(
(
)
)
)(
)(
(
)
(
)
)
)
(
(
(
)(
)(
)
(
)
)
)
Hence the required polynomial approximation to ( ) is given by ( )
(
)(
)
4.2.3 Errors in polynomial interpolation
)
Let the function ( ) defined by the (
) points (
differentiable
times and let ( ) be approximated by a polynomial
such that
be continuous and
( ) of degree not exceeding
( )
(4.17)
Now use ( ) to be approximate value of ( ) at some points other than those defined by (4.17).
Since the expression ( )
( ) vanishes for
we put
( )
( )
( )
(4.18)
Where
( )
And
(
)(
)
(
)
(4.19)
is to be obtained such that (4.18) hold for any intermediate value of , say
clearly
(
)
(
( )
8|Page
)
(4.20)
We construct a function ( ) such that
( )
( )
( )
( )
(4.21)
( )
( )
( )
Where is given in (4.20).It is clear that ( )
, that is ( )
vanished (n+2) times in the interval
; Consequently, by the repeated application of Rolle’s
theorem ( ) must vanish (n+1)times , ( ) must vanish n times , etc, in the interval
; In
particular, ( ) ( )must vanish once in the interval. Let this point be given by
differentiating (4.21) n+1times with respect to and putting
, we obtain
(
)
( )
(
) So that
(
. On
)
(
(4.22)
)
On comparison of (4.20) and (4.22), we get
( )
Where
( ) (
(
)
( )
)(
)
(4.23)
. This is the required expression for error.
4.2.4 Error in Lagrange’s interpolation formula
To determine the error of Lagrange’s interpolation formula for the class of functions which have
continuous derivatives of order up to (
) on
, we use equation (4.23)
( )
( )
( ) (
(
)
( )
)(
)
,
and the quantity
may be taken as estimate of error. Further if we assume that |
(
|
)
(
)
| ( )|
where
( )|
, then
( )|
( ) from the data given using Lagrange’s interpolation formula. Hence
Example 8 Find the value of
estimate the error in the solution
0
0
( )
Solution
Now, ( )
| ( )|
,
|
9|Page
(
)(
(
)(
)
(
)(
)
( )
)(
(
)
( )
)
|
(
)(
)
(
)(
)
( )
( )
(
)
, hence
( )
which agrees with the actual error in problem
, when
Example 9 Values of elliptic integral ( )
( )
for which
Find
√ ∫
are given below
√
( )
Solution By Lagrange’s inverse interpolation formula
(
(
)(
)(
)
)
(
(
(
(
)(
)(
)
)
(
(
)(
)(
(
(
)(
)(
)
)
)
)
(
(
)(
)(
)(
)(
)
)
)
)
4.2.4 Divided difference
Lagrange’s interpolation formula has the disadvantages that if any other interpolation point were
added, the interpolation coefficient will have to be recomputed. So an interpolation polynomial which
has the property that a polynomial of higher degree may be obtained from it by simply adding new
terms has to be derived. Newton’s general interpolation formula is one such formula and it employs
what are called divided difference.
Let
[
be the values of the function corresponding to values of the argument
which are equally spaced. Then the divided difference of order 1, 2, . . .,n are given as
]
[
]
[
]
and so on
[
The second order divided difference is given as
] [
]
[
] [
and so on.
The third order divided difference for
is given by
and the
order divided difference is
Now let the arguments be equally spaced so that
[
]
10 | P a g e
[
] [
, then
]
(
)
and in general
]
. If the tabulated function is a polynomial of nth degree,
would be a
th
constant and the n divided difference would also be constant.
Newton’s divided difference formula
From the definition of divided difference we’ve
; So that
(
Again
)
(4.24)
, which gives
(
)
(4.25)
From (4.24) and (4.25) we have
(
)([
]
Also
(
)
)
(4.26)
, which gives
(
)
)(
)
(
)(
(4.27)
From (4.26) and (4.27)
(
)
(
(
)(
)(
)
Proceeding in this manner, we get
( )
(
(
(
((
)
)(
)(
)(
)(
)
)(
)
)
)
(
(
)
))
(4.28)
This is called Newton’s general interpolation formula with divided difference the last term being the
remainder y term after (n+1) terms.
Newton’s divided difference formula can also written as
11 | P a g e
(
(
)
(
)(
)(
)(
)
)
(
(
)(
)(
)
)
(4.29)
are the 1st, the 2nd, the 3rd, . . . divided difference operators respectively
Where
Example 9 Construct a divided difference table for the following data
1 2
( ) 22 30
4
82
7
12
106 216
Solution
( )
1
22
2
30
4
82
( )
( )
( )
(
7
106
12
216
( )
)
Example 10 Using Newton’s divided difference formula calculate the value of ( ) from the following
data
1
2
7
8
( )
1
5
5
4
The divided difference table is
1
( )
1
2
5
( )
( )
( )
4
0
7
5
8
4
Applying Newton’s divided difference formula,
(
)
(
( )
12 | P a g e
(
)
)(
(
)
)(
(
)
)
(
(
)(
)(
)(
(
)(
)(
)
)
)
Exercises
1. Applying Lagrange’s formula find cubic polynomial which approximate the following data
( )
2. From the following table find
2
3
3
5
for which
2.6
2.8
6.695 8.198
3. The following table is given
0
( ) 2
1
3
3.0
10.018
2
12
5
147
What is the form of the function?
4. Using Newton’s divided difference method compute ( )from the following table
0
1
2
4
5
( ) 1
14
15
5
6
5. The following table gives the value of and
1.2
2.1
2.8
4.2
6.8
9.8
Find the value of corresponding to
13 | P a g e
4.1
13.4
6
19
4.9
6.2
15.5
19.6
using Lagrange’s technique of inverse interpolation