Download MATH 685/CSI 700 Lecture Notes

Numerical Analysis
Intro to Scientific Computing
Numerical Methods
Numerical Methods:
Algorithms that are used to obtain numerical solutions
of a mathematical problem.
Why do we need them?
1. No analytical solution exists,
2. An analytical solution is difficult to obtain
or not practical.
Why use Numerical Methods?

To solve problems that cannot be solved exactly
1
2
x
e


2
u

2
du
Introduction
1. Introduction to numerical methods for engineering as
a general and fundamental tool for all engineering
disciplines. We plan to cover (almost) the main
topics of numerical analysis.
2. We will use commercial software widely used in
science and engineering: MATLAB and Excel.
3. We will illustrate and discuss how numerical methods
are used in practice. We will consider examples from
Engineering.
Our choice for this course: Matlab

Matlab: numerical development environment.







Easy and fast programming
data types (vectors, matrices, complex numbers)
Complete functionality
Powerful toolkits
Campus license (from home: need Internet conn.)
But: €xpensive!
Alternatives:



Matlab student license (without toolkits)
Octave (free, Windows, Linux): www.octave.org
SciLab (free, Windows, Linux): www.scilab.org
Matlab basics

Variables are just assigned (no typedef needed)
a=42
s= 'test'

basic operators ( + - * / \ ^)
5/2

ans = 2.5000
functions (help elfun)
sqrt(3), sin(pi), cos(0)

ans is a system variable
pi is a system variable
display & clear variables
disp(a), disp('hello')
who, whos
clear a
clear
display value of a , “hello”
show all defined variables
clear variable a
clear all variables
arrow up/down keys recall your last commands
Matlab examples

Variables & Operators
a=5*(2/3)+3^2
a=2/4 + 4\2
a

result is shown
result is not shown
value of a is shown
;
Elementary functions
overview: doc elfun
abs(-1), sqrt(2)
tan(0), cos(0), acos(1) …
exp(2), log(1), log10(1) …

Rounding

round(2.3), round(2.5)
floor(5.7), floor(-1.2)
ceil (1.1), ceil(-2.7)
fix(1.7), fix (-2,7)

1
2
3
5
2
-2
-2
towards smaller
-2
towards larger
towards 0
Complex numbers
(2+3i) * (1i)
norm(1+1i)
-3+2i
1.4142
Modelling in Industry: Automobiles
8
Example of Solving an
Engineering Problem
http://numericalmethods.eng.usf.edu
9
Modelling in Industry: Aerospace
10
Modelling in Industry: Electronics
11
Course overview
1. Finding roots of functions of one
variable
2. Approximation, errors, and precision.
3. System of linear equations
4. Numerical integration and
differentiation.
Introduction
Why are Numerical Methods so widely used in
Engineering?
 Engineers use mathematical modeling (equations and
data) to describe and predict the behavior of systems.
 Closed-form (analytical) solutions are only possible
and complete for simple problems (geometry,
properties, etc.).
 Computers are widely available, powerful, and
(relatively) cheap.
 Powerful software packages are available (special or
general purpose).
Applications of Numerical
Methods in Engineering
• Communication/power
Network simulation
Train and traffic networks
• Computational Fluid Dynamics (CFD):
Weather prediction
Groundwater & pollutant movement
Electronic Communication by e-mail
• Computer assignments will be submitted as
attachments via e-mail:
[email protected]
• Text files, Excel & MATLAB documents as
attachments.
• documents will be distributed via the AAST
web page.
Useful info

Course website:

MATLAB instructions:
http://math.gmu.edu/introtomatlab.htm
Mathworks, the creator of MATLAB:
http://www.mathworks.com
OCTAVE = free MATLAB clone
Available for download at
http://octave.sourceforge.net/


Computational problems:
attack strategy

Develop mathematical model (usually requires a combination of
math skills and some a priori knowledge of the system)

Come up with numerical algorithm (numerical analysis skills)

Implement the algorithm (software skills)
Mathematical modeling

Run, debug, test the software

Visualize the results

Interpret and validate the results
Computational problems:
well-posedness

The problem is well-posed, if
(a) solution exists
(b) it is unique
(c) it depends continuously on problem data
Simplification strategies:
Infinite
finite
Nonlinear
linear
High-order
low-order
Sources of numerical errors



Before computation
 modeling approximations
 empirical measurements, human errorsCannot be controlled
 previous computations
During computation
Can be controlled through
 truncation or discretization
error analysis
 Rounding errors
Perturbations during computation may be amplified by
algorithm
Abs_error = approx_value – true_value
Rel_error = abs_error/true_value
Approx_value = (true_value)x(1+rel_error)
Representing Real Numbers

You are familiar with the decimal system:
312.45  3 10 2  1101  2 100  4 10 1  5 10 2

Decimal System: Base = 10 , Digits (0,1,…,9)

Standard Representations:

sign
3 1 2 . 4 5
integral
fraction
part
part
Normalized Floating Point
Representation

Normalized Floating Point Representation:
 d . f1 f 2 f 3 f 4  10 n
sign
mantissa
exponent
d  0,
 n : signed exponent

Scientific Notation: Exactly one non-zero digit appears before
decimal point.

Advantage: Efficient in representing very small or very large
numbers.
Binary System

Binary System:
Base = 2, Digits {0,1}
 1. f1 f 2 f 3 f 4  2  n
sign
mantissa
signed exponent
(1.101)2  (1  1  2 1  0  2  2  1  2 3 )10  (1.625)10
IEEE 754 Floating-Point
Standard

Single Precision (32-bit representation)

1-bit Sign + 8-bit Exponent + 23-bit Fraction
S Exponent8

Fraction23
Double Precision (64-bit representation)

1-bit Sign + 11-bit Exponent + 52-bit Fraction
S
Exponent11
Fraction52
(continued)
Machine precision
Calculator Example

Suppose you want to compute:
3.578 * 2.139
using a calculator with two-digit fractions
3.57
*
2.13 = 7.60
True answer:
7.653342
Stability

Algorithm is stable if result produced is relatively
insensitive to perturbations during computation

Stability of algorithms is analogous to
conditioning of
problems

For stable algorithm, effect of computational
error is no worse than effect of small data error
in input
Accuracy

Accuracy : closeness of computed
solution to true solution
of problem

Accuracy depends on conditioning of
problem as well as
stability of algorithm
Significant Digits - Example
48.9
Rounding and Chopping

Rounding: Replace the number by the nearest
machine number.

Chopping: Throw all extra digits.
Error Definitions – True Error
Can be computed if the true value is known:
Absolute True Error
Et  true value  approximat ion
Absolute Percent Relative Error
true value  approximat ion
t 
*100
true value
Notation
We say that the estimate is correct to n decimal
digits if:
Error  10
n
We say that the estimate is correct to n decimal
digits rounded if:
1
n
Error   10
2
Solution of Nonlinear
Equations

Some simple equations can be solved analytically:
x2  4x  3  0
Analytic solution roots 
4
4 2  4(1)(3)
2(1)
x  1 and x  3

Many other equations have no analytical solution:
9
2

x  2 x  5  0
 No analytic solution
x

xe

Methods for Solving Nonlinear
Equations
o
Bisection Method
o
Newton-Raphson Method
o
Secant Method
Solution of Systems of Linear
Equations
x1  x2  3
x1  2 x2  5
We can solve it as :
x1  3  x2 ,
3  x2  2 x2  5
 x2  2, x1  3  2  1
What to do if we have
1000 equations in 1000 unknowns.
Methods for Solving Systems
of Linear Equations
o
Gaussian Elimination
o
Gaussian Elimination with Scaled
Partial Pivoting
o
Gauss- Jordan
Integration

Some functions can be integrated
analytically:
3
3
1 2
9 1
1 xdx  2 x 1  2  2  4
But many functions have no analytical solutions :
a
e
0
 x2
dx  ?
Methods for Numerical
Integration
o
Trapezoid Method
o
Simpson Method
o
Mid-point method