Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
List of important publications in mathematics wikipedia , lookup
Positional notation wikipedia , lookup
Numerical continuation wikipedia , lookup
Karhunen–Loève theorem wikipedia , lookup
Approximations of π wikipedia , lookup
System of polynomial equations wikipedia , lookup
Elementary mathematics wikipedia , lookup
0761214: Numerical Analysis Topic 1: Introduction to Numerical Methods and Taylor Series Lectures 1-4: 0761214_Topic1 1 Lecture 1 Introduction to Numerical Methods What are NUMERICAL METHODS? Why do we need them? Topics covered in 0761214. Reading Assignment: Pages 3-10 of textbook 0761214_Topic1 2 Numerical Methods Numerical Methods: Algorithms that are used to obtain numerical solutions of a mathematical problem. Why do we need them? 1. No analytical solution exists, 2. An analytical solution is difficult to obtain or not practical. 0761214_Topic1 3 What do we need? Basic Needs in the Numerical Methods: Practical: Can be computed in a reasonable amount of time. Accurate: 0761214_Topic1 Good approximate to the true value, Information about the approximation error (Bounds, error order,… ). 4 Outlines of the Course Taylor Theorem Number Representation Solution of nonlinear Equations Interpolation Numerical Differentiation Numerical Integration 0761214_Topic1 Solution of linear Equations Least Squares curve fitting Solution of ordinary differential equations Solution of Partial differential equations 5 Solution of Nonlinear Equations Some simple equations can be solved analytically: x2 4x 3 0 Analy ticsolution roots 4 4 2 4(1)(3) 2(1) x 1 and x 3 Many other equations have no analytical solution: x 9 2 x 2 5 0 No analy tic solution x xe 0761214_Topic1 6 Methods for Solving Nonlinear Equations o Bisection Method o Newton-Raphson Method o Secant Method 0761214_Topic1 7 Solution of Systems of Linear Equations x1 x2 3 x1 2 x2 5 We can solve it as : x1 3 x2 , 3 x 2 2 x2 5 x2 2, x1 3 2 1 What to do if we have 1000 equations in 1000 unknowns. 0761214_Topic1 8 Cramer’s Rule is Not Practical Cramer' s Rule can be used to solve the system: 3 5 x1 1 1 1 2 1, 1 2 1 1 x2 1 1 3 5 2 1 2 But Cramer' s Rule is not practical for large problems. To solve N equations with N unknowns, we need (N 1)(N 1)N! multiplications. To solve a 30 by 30 system,2.3 1035 multiplications are needed. A super computer needs more than 10 20 years to compute this. 0761214_Topic1 9 Methods for Solving Systems of Linear Equations o Naive Gaussian Elimination o Gaussian Elimination with Scaled Partial Pivoting o Algorithm for Tri-diagonal Equations 0761214_Topic1 10 Curve Fitting Given a set of data: x 0 1 2 y 0.5 10.3 21.3 Select a curve that best fits the data. One choice is to find the curve so that the sum of the square of the error is minimized. 0761214_Topic1 11 Interpolation Given a set of data: xi 0 1 2 yi 0.5 10.3 15.3 Find a polynomial P(x) whose graph passes through all tabulated points. yi P( xi ) if xi is in the table 0761214_Topic1 12 Methods for Curve Fitting o Least Squares o o o Linear Regression Nonlinear Least Squares Problems Interpolation o o Newton Polynomial Interpolation Lagrange Interpolation 0761214_Topic1 13 Integration Some functions can be integrated analytically: 3 3 1 2 9 1 1 xdx 2 x 1 2 2 4 But many functions have no analytical solutions : a e x2 dx ? 0 0761214_Topic1 14 Methods for Numerical Integration o Upper and Lower Sums o Trapezoid Method o Romberg Method o Gauss Quadrature 0761214_Topic1 15 Solution of Ordinary Differential Equations A solution to the differential equation : x' ' (t ) 3 x' (t ) 3 x(t ) 0 x' (0) 1; x(0) 0 is a function x(t) that satisfies the equations. * Analytical solutions are available for special cases only. 0761214_Topic1 16 Solution of Partial Differential Equations Partial Differential Equations are more difficult to solve than ordinary differential equations: u 2 u 2 20 x t u (0, t ) u (1, t ) 0, u ( x,0) sin(x) 2 0761214_Topic1 2 17 Summary Numerical Methods: Algorithms that are used to obtain numerical solution of a mathematical problem. We need them when No analytical solution exists or it is difficult to obtain it. Topics Covered in the Course 0761214_Topic1 Solution of Nonlinear Equations Solution of Linear Equations Curve Fitting Least Squares Interpolation Numerical Integration Numerical Differentiation Solution of Ordinary Differential Equations Solution of Partial Differential Equations 18 Lecture 2 Number Representation and Accuracy Number Representation Normalized Floating Point Representation Significant Digits Accuracy and Precision Rounding and Chopping Reading Assignment: Chapter 3 0761214_Topic1 19 Representing Real Numbers You are familiar with the decimal system: 312.45 3 102 1101 2 100 4 101 5 102 Decimal System: Base = 10 , Digits (0,1,…,9) Standard Representations: 3 1 2 . 4 5 sign integral fraction part part 0761214_Topic1 20 Normalized Floating Point Representation Normalized Floating Point Representation: d . f1 f 2 f 3 f 4 10 n sign mantissa exponent d 0, n : signed exponent Scientific Notation: Exactly one non-zero digit appears before decimal point. Advantage: Efficient in representing very small or very large numbers. 0761214_Topic1 21 Binary System Binary System: Base = 2, Digits {0,1} 1. f1 f 2 f 3 f 4 2 n sign mantissa signed exponent (1.101)2 (1 1 21 0 22 1 23 )10 (1.625)10 0761214_Topic1 22 Fact Numbers that have a finite expansion in one numbering system may have an infinite expansion in another numbering system: (1.1)10 (1.000110011001100...) 2 You can never represent 1.1 exactly in binary system. 0761214_Topic1 23 IEEE 754 Floating-Point Standard Single Precision (32-bit representation) 1-bit Sign + 8-bit Exponent + 23-bit Fraction S Exponent8 Fraction23 Double Precision (64-bit representation) 1-bit Sign + 11-bit Exponent + 52-bit Fraction S Exponent11 Fraction52 (continued) 0761214_Topic1 24 Significant Digits Significant digits are those digits that can be used with confidence. Single-Precision: 7 Significant Digits 1.175494… × 10-38 to 3.402823… × 1038 Double-Precision: 15 Significant Digits 2.2250738… × 10-308 to 1.7976931… × 10308 0761214_Topic1 25 Remarks Numbers that can be exactly represented are called machine numbers. Difference between machine numbers is not uniform Sum of machine numbers is not necessarily a machine number 0761214_Topic1 26 Calculator Example Suppose you want to compute: 3.578 * 2.139 using a calculator with two-digit fractions 3.57 * 2.13 = 7.60 True answer: 0761214_Topic1 7.653342 27 Significant Digits - Example 48.9 0761214_Topic1 28 Accuracy and Precision Accuracy is related to the closeness to the true value. Precision is related to the closeness to other estimated values. 0761214_Topic1 29 0761214_Topic1 30 Rounding and Chopping Rounding: Replace the number by the nearest machine number. Chopping: Throw all extra digits. 0761214_Topic1 31 Rounding and Chopping 0761214_Topic1 32 Error Definitions – True Error Can be computed if the true value is known: Absolute True Error Et true value approximation Absolute Percent Relative Error true value approximation t *100 true value 0761214_Topic1 33 Error Definitions – Estimated Error When the true value is not known: Estimated Absolute Error Ea current estimate previous estimate Estimated Absolute Percent Relative Error current estimate previous estimate a *100 current estimate 0761214_Topic1 34 Notation We say that the estimate is correct to n decimal digits if: Error 10 n We say that the estimate is correct to n decimal digits rounded if: 1 n Error 10 2 0761214_Topic1 35 Summary Number Representation Numbers that have a finite expansion in one numbering system may have an infinite expansion in another numbering system. Normalized Floating Point Representation Efficient in representing very small or very large numbers, Difference between machine numbers is not uniform, Representation error depends on the number of bits used in the mantissa. 0761214_Topic1 36 Lectures 3-4 Taylor Theorem Motivation Taylor Theorem Examples Reading assignment: Chapter 4 0761214_Topic1 37 Motivation We can easily compute expressions like: 3 10 2 2( x 4) But, How do you compute Can we use the definition to compute sin(0.6)? Is this a practical way? 0761214_Topic1 4.1, sin(0.6) ? b a 0.6 38 Remark In this course, all angles are assumed to be in radian unless you are told otherwise. 0761214_Topic1 39 Taylor Series The Taylor series expansion of f ( x ) about a : f ( 3) ( a ) f ( 2) (a ) 2 ( x a ) 3 ... ( x a) f (a ) f (a ) ( x a ) 3! 2! or ' Taylor Series k 0 1 (k ) f (a ) ( x a )k k! If the series converge, we can write : ∞ f ( x) 0761214_Topic1 1 (k ) ∑k! f (a ) ( x a )k k 0 40 Maclaurin Series Maclaurin series is a special case of Taylor series with the center of expansion a = 0. The Maclaurin series expansion of f ( x ) : ( 2) ( 3) f ( 0 ) f ( 0) 3 ' 2 f ( 0) f ( 0) x x x ... 2! 3! If the series converge, we can write : ∞ 1 (k ) f ( x ) ∑ f ( 0) x k k! k 0 0761214_Topic1 41 Maclaurin Series – Example 1 Obtain Maclaurin series expansion of f ( x ) e x f ( x) e x f ( 0) 1 f ' ( x) e x f ' ( 0) 1 f ( 2) ( x ) e x f ( 2 ) ( 0) 1 f (k ) ( x) e x f ( k ) (0) 1 for k 1 ∞ ∞ 1 (k ) xk x2 x3 x k e ∑ f ( 0) x ∑ 1 x ... k! k! 2! 3! k 0 k 0 The series converges for x ∞. 0761214_Topic1 42 Taylor Series 3 Example 1 2.5 exp(x) 1+x+0.5x 2 2 1+x 1.5 1 1 0.5 0 -1 0761214_Topic1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 43 Maclaurin Series – Example 2 Obtain Maclaurin series expansion of f ( x) sin( x ) : f ( x ) sin( x ) f ' ( x ) cos( x ) f ( 0) 0 f ' ( 0) 1 f ( 2 ) ( x ) sin( x ) f ( 2 ) ( 0) 0 f ( 3) ( x ) cos( x ) f ( 3) (0) 1 ∞ f ( k ) ( 0) k x3 x5 x7 sin( x ) ∑ x x .... k! 3! 5! 7! k 0 The series converges for x ∞. 0761214_Topic1 44 4 3 x 2 1 x-x 3/3!+x 5/5! 0 sin(x) -1 x-x 3/3! -2 -3 -4 -4 0761214_Topic1 -3 -2 -1 0 1 2 3 4 45 Maclaurin Series – Example 3 Obtain Maclaurin series expansion of : f ( x) cos( x) f ( x ) cos( x ) f ' ( x ) sin( x ) f ( 0) 1 f ' ( 0) 0 f ( 2 ) ( x ) cos( x ) f ( 2 ) ( 0 ) 1 f ( 3) ( x ) sin( x ) f ( 3) (0) 0 ∞ 2 4 6 f ( k ) ( 0) x x x cos( x ) ∑ ( x ) k 1 .... k! 2! 4! 6! k 0 The series converges for x ∞. 0761214_Topic1 46 Maclaurin Series – Example 4 Obtain Maclaurin series expansion of f(x) 1 1 x 1 f ' ( x) 1 x 2 2 f ( 2) ( x ) 1 x 3 6 f ( 3) ( x ) 1 x 4 f ( x) 1 1 x f ( 0) 1 Maclaurin Series Expansion of : f ' ( 0) 1 f ( 2 ) ( 0) 2 f ( 3) (0) 6 1 1 x x 2 x 3 ... 1 x Series converges for | x | 1 0761214_Topic1 47 Example 4 - Remarks Can we apply the series for x≥1?? How many terms are needed to get a good approximation??? These questions will be answered using Taylor’s Theorem. 0761214_Topic1 48 Taylor Series – Example 5 Obtain Taylor series expansion of f(x) 1 x 1 f ' ( x) 2 x 2 f ( 2) ( x ) 3 x 6 f ( 3) ( x ) 4 x f ( x) 1 at a 1 x f (1) 1 f ' (1) 1 f ( 2 ) (1) 2 f ( 3) (1) 6 Taylor Series Expansion ( a 1) : 1 ( x 1) ( x 1) 2 ( x 1) 3 ... 0761214_Topic1 49 Taylor Series – Example 6 Obtain Taylor series expansion of f(x) ln( x ) at ( a 1) 1 1 2 ( 2) ( 3) f ( x ) ln( x ) , f ' ( x ) , f ( x ) 2 , f ( x ) 3 x x x f (1) 0, f ' (1) 1, f ( 2 ) (1) 1 f ( 3) (1) 2 1 2 1 Taylor Series Expansion : ( x 1) ( x 1) ( x 1) 3 ... 2 3 0761214_Topic1 50 Convergence of Taylor Series The Taylor series converges fast (few terms are needed) when x is near the point of expansion. If |x-a| is large then more terms are needed to get a good approximation. 0761214_Topic1 51 Taylor’s Theorem If a function f ( x ) possesses derivatives of orders 1, 2, ..., ( n 1) on an interval containing a and x then the value of f ( x ) is given by : (n+1) terms Truncated Taylor Series f ( x) n ∑ k 0 f ( k ) (a ) ( x a)k k! Rn Remainder where : f ( n 1) ( ) Rn ( x a ) n 1 and is between a and x. ( n 1)! 0761214_Topic1 52 Taylor’s Theorem We can apply Taylor' s theorem for : 1 f(x) with the point of expansion a 0 if | x | 1. 1 x If x 1, then the function and its derivatives are not defined. Taylor Theorem is not applicable . 0761214_Topic1 53 Error Term To get an idea about the approximation error, we can derive an upper bound on : ( n 1) ( ) Rn ( x a ) n 1 ( n 1)! for all values of between a and x. f 0761214_Topic1 54 Error Term - Example How large is the error if we replaced f ( x ) e by the first 4 terms ( n 3) of its Taylor series expansion at a 0 when x 0.2 ? x f (n) ( x) e x f ( n ) ( ) ≤ e 0.2 for n ≥ 1 f ( n 1) ( ) Rn ( x a ) n 1 ( n 1)! e 0.2 n 1 Rn 0.2 R3 8.14268E 05 ( n 1)! 0761214_Topic1 55 Alternative form of Taylor’s Theorem Let f ( x ) have derivatives of orders 1, 2, ..., ( n 1) on an interval containing x and x h then : f ( x h) n k 0 f (k ) ( x) k h Rn k! ( h step size) f ( n 1) ( ) n 1 Rn h where is between x and x h ( n 1)! 0761214_Topic1 56 Taylor’s Theorem – Alternative forms ( n 1) f ( k ) (a ) f ( ) k f ( x) ( x a) ( x a ) n 1 k! ( n 1)! k 0 n where is between a and x. a x, x x h f ( k ) ( x ) k f ( n 1) ( ) n 1 f ( x h) h h k! ( n 1)! k 0 n where is between x and x h. 0761214_Topic1 57 Mean Value Theorem If f ( x ) is a continuous function on a closed interval [a, b] and its derivative is defined on the open interval ( a, b) then there exists ξ ( a, b) f(b) f(a) f ' (ξ ) ba Proof : Use Taylor' s Theorem for n 0, x a, x h b f(b) f(a) f ' (ξ ) (b a ) 0761214_Topic1 58 Alternating Series Theorem Consider the alternating series : S a1 a2 a3 a4 a a a a 2 3 4 1 If and lim a 0 n n The series converges then and S S n an 1 S n : Partial sum (sum of the first n terms) an 1 : First omitted term 0761214_Topic1 59 Alternating Series – Example 1 1 1 sin(1) can be computed using : sin (1) 1 3! 5! 7! This is a convergent alternating series since : a1 a2 a3 a4 and lim an 0 n Then : 1 1 sin (1) 1 3! 5! 1 1 1 sin (1) 1 3! 5! 7! 0761214_Topic1 60 Example 7 Obtain the Taylor series expansion of f ( x ) e 2 x 1 at a 0.5 (the center of expansion) How large can the error be when ( n 1) terms are used to approximate e 2 x 1 with x 1 ? 0761214_Topic1 61 Example 7 – Taylor Series Obtain Taylor series expansion of f ( x ) e 2 x 1 , a 0.5 f ( x) e 2 x 1 f (0.5) e 2 f ' ( x) 2e 2 x 1 f ' (0.5) 2e 2 f ( 2) ( x) 4e 2 x 1 f ( k ) ( x) 2 k e 2 x 1 e 2 x 1 f ( 2) (0.5) 4e 2 f ( k ) (0.5) 2 k e 2 ∞ f ( k ) (0.5) ∑ ( x 0.5) k k! k 0 k ( x 0.5) 2 k 2 ( x 0.5) e 2e ( x 0.5) 4e ... 2 e ... 2! k! 2 0761214_Topic1 2 2 62 Example 7 – Error Term f ( k ) ( x) 2 k e 2 x 1 f ( n 1) ( ) Error ( x 0.5) n 1 (n 1)! Error 2 n 1 2 1 e (1 0.5) n 1 (n 1)! n 1 ( 0 . 5 ) Error 2 n 1 max e 2 1 (n 1)! [ 0.5,1] e3 Error (n 1)! 0761214_Topic1 63