Download Document

0761214: Numerical Analysis
Topic 1:
Introduction to Numerical Methods and Taylor Series
Lectures 1-4:
0761214_Topic1
1
Lecture 1
Introduction to Numerical Methods



What are NUMERICAL METHODS?
Why do we need them?
Topics covered in 0761214.
Reading Assignment: Pages 3-10 of textbook
0761214_Topic1
2
Numerical Methods
Numerical Methods:
Algorithms that are used to obtain numerical
solutions of a mathematical problem.
Why do we need them?
1. No analytical solution exists,
2. An analytical solution is difficult to obtain
or not practical.
0761214_Topic1
3
What do we need?
Basic Needs in the Numerical Methods:
 Practical:
Can be computed in a reasonable amount of time.

Accurate:


0761214_Topic1
Good approximate to the true value,
Information about the approximation error
(Bounds, error order,… ).
4
Outlines of the Course






Taylor Theorem
Number
Representation
Solution of nonlinear
Equations
Interpolation
Numerical
Differentiation
Numerical Integration
0761214_Topic1




Solution of linear
Equations
Least Squares curve
fitting
Solution of ordinary
differential equations
Solution of Partial
differential equations
5
Solution of Nonlinear Equations

Some simple equations can be solved analytically:
x2  4x  3  0
Analy ticsolution roots 
4
4 2  4(1)(3)
2(1)
x  1 and x  3

Many other equations have no analytical solution:
x 9  2 x 2  5  0

 No analy tic solution
x

xe

0761214_Topic1
6
Methods for Solving Nonlinear Equations
o
Bisection Method
o
Newton-Raphson Method
o
Secant Method
0761214_Topic1
7
Solution of Systems of Linear Equations
x1  x2  3
x1  2 x2  5
We can solve it as :
x1  3  x2 ,
3  x 2  2 x2  5
 x2  2, x1  3  2  1
What to do if we have
1000 equations in 1000 unknowns.
0761214_Topic1
8
Cramer’s Rule is Not Practical
Cramer' s Rule can be used to solve the system:
3
5
x1 
1
1
1
2
 1,
1
2
1
1
x2 
1
1
3
5
2
1
2
But Cramer' s Rule is not practical for large problems.
To solve N equations with N unknowns, we need (N  1)(N  1)N!
multiplications.
To solve a 30 by 30 system,2.3 1035 multiplications are needed.
A super computer needs more than 10 20 years to compute this.
0761214_Topic1
9
Methods for Solving Systems of Linear
Equations
o
Naive Gaussian Elimination
o
Gaussian Elimination with Scaled
Partial Pivoting
o
Algorithm for Tri-diagonal
Equations
0761214_Topic1
10
Curve Fitting


Given a set of data:
x
0
1
2
y
0.5
10.3
21.3
Select a curve that best fits the data. One
choice is to find the curve so that the sum
of the square of the error is minimized.
0761214_Topic1
11
Interpolation


Given a set of data:
xi
0
1
2
yi
0.5
10.3
15.3
Find a polynomial P(x) whose graph
passes through all tabulated points.
yi  P( xi ) if xi is in the table
0761214_Topic1
12
Methods for Curve Fitting
o
Least Squares
o
o
o
Linear Regression
Nonlinear Least Squares Problems
Interpolation
o
o
Newton Polynomial Interpolation
Lagrange Interpolation
0761214_Topic1
13
Integration

Some functions can be integrated
analytically:
3
3
1 2
9 1
1 xdx  2 x 1  2  2  4
But many functions have no analytical solutions :
a
e
 x2
dx  ?
0
0761214_Topic1
14
Methods for Numerical Integration
o
Upper and Lower Sums
o
Trapezoid Method
o
Romberg Method
o
Gauss Quadrature
0761214_Topic1
15
Solution of Ordinary Differential Equations
A solution to the differential equation :
x' ' (t )  3 x' (t )  3 x(t )  0
x' (0)  1; x(0)  0
is a function x(t) that satisfies the equations.
* Analytical solutions are available for
special cases only.
0761214_Topic1
16
Solution of Partial Differential Equations
Partial Differential Equations are more
difficult to solve than ordinary differential
equations:
 u
2

 u
2
20
x
t
u (0, t )  u (1, t )  0, u ( x,0)  sin(x)
2
0761214_Topic1
2
17
Summary


Numerical Methods:
Algorithms that are
used to obtain
numerical solution of a
mathematical problem.
We need them when
No analytical solution
exists or it is difficult
to obtain it.
Topics Covered in the Course









0761214_Topic1
Solution of Nonlinear Equations
Solution of Linear Equations
Curve Fitting
Least Squares
Interpolation
Numerical Integration
Numerical Differentiation
Solution of Ordinary Differential
Equations
Solution of Partial Differential
Equations
18
Lecture 2
Number Representation and Accuracy





Number Representation
Normalized Floating Point Representation
Significant Digits
Accuracy and Precision
Rounding and Chopping
Reading Assignment: Chapter 3
0761214_Topic1
19
Representing Real Numbers

You are familiar with the decimal system:
312.45  3 102  1101  2 100  4 101  5 102

Decimal System:
Base = 10 , Digits (0,1,…,9)

Standard Representations:
 3 1 2 . 4 5
sign integral
fraction
part
part
0761214_Topic1
20
Normalized Floating Point Representation

Normalized Floating Point Representation:
 d . f1 f 2 f 3 f 4  10 n
sign
mantissa
exponent
d  0,
 n : signed exponent

Scientific Notation: Exactly one non-zero digit appears
before decimal point.

Advantage: Efficient in representing very small or very
large numbers.
0761214_Topic1
21
Binary System

Binary System:
Base = 2, Digits {0,1}
 1. f1 f 2 f 3 f 4  2  n
sign
mantissa
signed exponent
(1.101)2  (1  1  21  0  22  1  23 )10  (1.625)10
0761214_Topic1
22
Fact

Numbers that have a finite expansion in one numbering
system may have an infinite expansion in another
numbering system:
(1.1)10  (1.000110011001100...) 2

You can never represent 1.1 exactly in binary system.
0761214_Topic1
23
IEEE 754 Floating-Point Standard

Single Precision (32-bit representation)

1-bit Sign + 8-bit Exponent + 23-bit Fraction
S Exponent8

Fraction23
Double Precision (64-bit representation)

1-bit Sign + 11-bit Exponent + 52-bit Fraction
S
Exponent11
Fraction52
(continued)
0761214_Topic1
24
Significant Digits

Significant digits are those digits that can be
used with confidence.

Single-Precision: 7 Significant Digits
1.175494… × 10-38 to 3.402823… × 1038

Double-Precision: 15 Significant Digits
2.2250738… × 10-308 to 1.7976931… × 10308
0761214_Topic1
25
Remarks

Numbers that can be exactly represented are called
machine numbers.

Difference between machine numbers is not uniform

Sum of machine numbers is not necessarily a machine
number
0761214_Topic1
26
Calculator Example

Suppose you want to compute:
3.578 * 2.139
using a calculator with two-digit fractions
3.57
*
2.13 = 7.60
True answer:
0761214_Topic1
7.653342
27
Significant Digits - Example
48.9
0761214_Topic1
28
Accuracy and Precision

Accuracy is related to the closeness to the true
value.

Precision is related to the closeness to other
estimated values.
0761214_Topic1
29
0761214_Topic1
30
Rounding and Chopping

Rounding: Replace the number by the nearest
machine number.

Chopping: Throw all extra digits.
0761214_Topic1
31
Rounding and Chopping
0761214_Topic1
32
Error Definitions – True Error
Can be computed if the true value is known:
Absolute True Error
Et  true value  approximation
Absolute Percent Relative Error
true value  approximation
t 
*100
true value
0761214_Topic1
33
Error Definitions – Estimated Error
When the true value is not known:
Estimated Absolute Error
Ea  current estimate  previous estimate
Estimated Absolute Percent Relative Error
current estimate  previous estimate
a 
*100
current estimate
0761214_Topic1
34
Notation
We say that the estimate is correct to n
decimal digits if:
Error  10
n
We say that the estimate is correct to n
decimal digits rounded if:
1
n
Error   10
2
0761214_Topic1
35
Summary

Number Representation
Numbers that have a finite expansion in one numbering system
may have an infinite expansion in another numbering system.

Normalized Floating Point Representation



Efficient in representing very small or very large numbers,
Difference between machine numbers is not uniform,
Representation error depends on the number of bits used in
the mantissa.
0761214_Topic1
36
Lectures 3-4
Taylor Theorem



Motivation
Taylor Theorem
Examples
Reading assignment: Chapter 4
0761214_Topic1
37
Motivation

We can easily compute expressions like:
3 10 2
2( x  4)
But, How do you compute
Can we use the definition
to compute sin(0.6)?
Is this a practical way?
0761214_Topic1
4.1, sin(0.6) ?
b
a
0.6
38
Remark

In this course, all angles are assumed to
be in radian unless you are told otherwise.
0761214_Topic1
39
Taylor Series
The Taylor series expansion of f ( x ) about a :
f ( 3) ( a )
f ( 2) (a )
2
( x  a ) 3  ...
( x  a) 
f (a )  f (a ) ( x  a ) 
3!
2!
or
'
Taylor Series 


k 0
1 (k )
f (a ) ( x  a )k
k!
If the series converge, we can write :
∞
f ( x) 
0761214_Topic1
1 (k )
∑k! f (a ) ( x  a )k
k 0
40
Maclaurin Series

Maclaurin series is a special case of Taylor
series with the center of expansion a = 0.
The Maclaurin series expansion of f ( x ) :
( 2)
( 3)
f
(
0
)
f
( 0) 3
'
2
f ( 0)  f ( 0) x 
x 
x  ...
2!
3!
If the series converge, we can write :
∞
1 (k )
f ( x )  ∑ f ( 0) x k
k!
k 0
0761214_Topic1
41
Maclaurin Series – Example 1
Obtain Maclaurin series expansion of f ( x )  e x
f ( x)  e x
f ( 0)  1
f ' ( x)  e x
f ' ( 0)  1
f ( 2) ( x )  e x
f ( 2 ) ( 0)  1
f (k ) ( x)  e x
f ( k ) (0)  1 for k  1
∞
∞
1 (k )
xk
x2 x3
x
k
e  ∑ f ( 0) x  ∑
 1 x 

 ...
k!
k!
2! 3!
k 0
k 0
The series converges for x  ∞.
0761214_Topic1
42
Taylor Series
3
Example 1
2.5
exp(x)
1+x+0.5x 2
2
1+x
1.5
1
1
0.5
0
-1
0761214_Topic1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
43
Maclaurin Series – Example 2
Obtain Maclaurin series expansion of f ( x)  sin( x ) :
f ( x )  sin( x )
f ' ( x )  cos( x )
f ( 0)  0
f ' ( 0)  1
f ( 2 ) ( x )   sin( x )
f ( 2 ) ( 0)  0
f ( 3) ( x )   cos( x )
f ( 3) (0)  1
∞
f ( k ) ( 0) k
x3 x5 x7
sin( x )  ∑
x  x     ....
k!
3! 5! 7!
k 0
The series converges for x  ∞.
0761214_Topic1
44
4
3
x
2
1
x-x 3/3!+x 5/5!
0
sin(x)
-1
x-x 3/3!
-2
-3
-4
-4
0761214_Topic1
-3
-2
-1
0
1
2
3
4
45
Maclaurin Series – Example 3
Obtain Maclaurin series expansion of : f ( x)  cos( x)
f ( x )  cos( x )
f ' ( x )   sin( x )
f ( 0)  1
f ' ( 0)  0
f ( 2 ) ( x )   cos( x )
f ( 2 ) ( 0 )  1
f ( 3) ( x )  sin( x )
f ( 3) (0)  0
∞
2
4
6
f ( k ) ( 0)
x
x
x
cos( x )  ∑
( x ) k  1     ....
k!
2! 4! 6!
k 0
The series converges for x  ∞.
0761214_Topic1
46
Maclaurin Series – Example 4
Obtain Maclaurin series expansion of f(x) 
1
1 x
1
f ' ( x) 
1  x 2
2
f ( 2) ( x ) 
1  x 3
6
f ( 3) ( x ) 
1  x 4
f ( x) 
1
1 x
f ( 0)  1
Maclaurin Series Expansion of :
f ' ( 0)  1
f ( 2 ) ( 0)  2
f ( 3) (0)  6
1
 1  x  x 2  x 3  ...
1 x
Series converges for | x |  1
0761214_Topic1
47
Example 4 - Remarks

Can we apply the series for x≥1??

How many terms are needed to get a good
approximation???
These questions will be answered using
Taylor’s Theorem.
0761214_Topic1
48
Taylor Series – Example 5
Obtain Taylor series expansion of f(x) 
1
x
1
f ' ( x)  2
x
2
f ( 2) ( x )  3
x
6
f ( 3) ( x )  4
x
f ( x) 
1
at a  1
x
f (1)  1
f ' (1)  1
f ( 2 ) (1)  2
f ( 3) (1)  6
Taylor Series Expansion ( a  1) : 1  ( x  1)  ( x  1) 2  ( x  1) 3  ...
0761214_Topic1
49
Taylor Series – Example 6
Obtain Taylor series expansion of f(x)  ln( x ) at ( a  1)
1
1
2
( 2)
( 3)
f ( x )  ln( x ) , f ' ( x )  , f ( x )  2 , f ( x )  3
x
x
x
f (1)  0,
f ' (1)  1,
f ( 2 ) (1)  1
f ( 3) (1)  2
1
2 1
Taylor Series Expansion : ( x  1)  ( x  1)  ( x  1) 3  ...
2
3
0761214_Topic1
50
Convergence of Taylor Series

The Taylor series converges fast (few terms
are needed) when x is near the point of
expansion. If |x-a| is large then more terms
are needed to get a good approximation.
0761214_Topic1
51
Taylor’s Theorem
If a function f ( x ) possesses derivatives of orders 1, 2, ..., ( n  1)
on an interval containing a and x then the value of f ( x ) is given by :
(n+1) terms Truncated
Taylor Series
f ( x) 
n
∑
k 0
f ( k ) (a )
( x  a)k
k!
 Rn
Remainder
where :
f ( n 1) ( )
Rn 
( x  a ) n 1 and  is between a and x.
( n  1)!
0761214_Topic1
52
Taylor’s Theorem
We can apply Taylor' s theorem for :
1
f(x) 
with the point of expansion a  0 if | x |  1.
1 x
If x  1, then the function and its
derivatives are not defined.
 Taylor Theorem is not applicable .
0761214_Topic1
53
Error Term
To get an idea about the approximation error,
we can derive an upper bound on :
( n 1)
( )
Rn 
( x  a ) n 1
( n  1)!
for all values of  between a and x.
f
0761214_Topic1
54
Error Term - Example
How large is the error if we replaced f ( x )  e by
the first 4 terms ( n  3) of its Taylor series expansion
at a  0 when x  0.2 ?
x
f (n) ( x)  e x
f ( n ) ( ) ≤ e 0.2 for n ≥ 1
f ( n 1) ( )
Rn 
( x  a ) n 1
( n  1)!
e 0.2
n 1


Rn 
0.2
 R3  8.14268E  05
( n  1)!
0761214_Topic1
55
Alternative form of Taylor’s Theorem
Let f ( x ) have derivatives of orders 1, 2, ..., ( n  1)
on an interval containing x and x  h then :
f ( x  h) 
n

k 0
f
(k )
( x) k
h  Rn
k!
( h  step size)
f ( n 1) ( ) n 1
Rn 
h
where  is between x and x  h
( n  1)!
0761214_Topic1
56
Taylor’s Theorem – Alternative forms
( n 1)
f ( k ) (a )
f
( )
k
f ( x)  
( x  a) 
( x  a ) n 1
k!
( n  1)!
k 0
n
where  is between a and x.
a  x, x  x  h
f ( k ) ( x ) k f ( n 1) ( ) n 1
f ( x  h)  
h 
h
k!
( n  1)!
k 0
n
where  is between x and x  h.
0761214_Topic1
57
Mean Value Theorem
If f ( x ) is a continuous function on a closed interval [a, b]
and its derivative is defined on the open interval ( a, b)
then there exists ξ  ( a, b)
f(b)  f(a)
f ' (ξ ) 
ba
Proof : Use Taylor' s Theorem for n  0, x  a, x  h  b
f(b)  f(a)  f ' (ξ ) (b  a )
0761214_Topic1
58
Alternating Series Theorem
Consider the alternating series :
S  a1  a2  a3  a4  
 a  a  a  a 
2
3
4
 1
If  and
 lim a  0
 n n
The series converges

then 
and

S  S n  an 1

S n : Partial sum (sum of the first n terms)
an 1 : First omitted term
0761214_Topic1
59
Alternating Series – Example
1 1 1
sin(1) can be computed using : sin (1)  1     
3! 5! 7!
This is a convergent alternating series since :
a1  a2  a3  a4   and lim an  0
n 
Then :
 1 1
sin (1)  1   
 3!  5!
 1 1 1
sin (1)  1    
 3! 5!  7!
0761214_Topic1
60
Example 7
Obtain the Taylor series expansion
of f ( x )  e 2 x 1 at a  0.5 (the center of expansion)
How large can the error be when ( n  1) terms are used
to approximate e 2 x 1 with x  1 ?
0761214_Topic1
61
Example 7 – Taylor Series
Obtain Taylor series expansion of f ( x )  e 2 x 1 , a  0.5
f ( x)  e 2 x 1
f (0.5)  e 2
f ' ( x)  2e 2 x 1
f ' (0.5)  2e 2
f ( 2) ( x)  4e 2 x 1
f ( k ) ( x)  2 k e 2 x 1
e
2 x 1
f ( 2) (0.5)  4e 2
f ( k ) (0.5)  2 k e 2
∞
f ( k ) (0.5)
∑
( x  0.5) k
k!
k 0
k
( x  0.5) 2
k 2 ( x  0.5)
 e  2e ( x  0.5)  4e
 ...  2 e
 ...
2!
k!
2
0761214_Topic1
2
2
62
Example 7 – Error Term
f ( k ) ( x)  2 k e 2 x 1
f ( n 1) ( )
Error 
( x  0.5) n 1
(n  1)!
Error  2
n 1 2 1
e
(1  0.5) n 1
(n  1)!
n 1
(
0
.
5
)
Error  2 n 1
max e 2 1
(n  1)!  [ 0.5,1]
e3
Error 
(n  1)!
0761214_Topic1
63