Download Presentation

Statistical &
Uncertainty Analysis
UC Summer REU Programs
Instructor: Lilit Yeghiazarian, PhD
Environmental Engineering Program
1
Instructor
Dr. Lilit Yeghiazarian
Environmental Engineering Program
Office: 746 Engineering Research Center (ERC)
Email: [email protected]
Phone: 513-556-3623
2
Textbooks
Applied Numerical Methods with MATLAB
for Engineers and Scientists, 3rd edition,
S.C. Chapra, McGraw-Hill Companies,
Inc., 2012
 An Introduction to Error Analysis:
The Study of Uncertainties in Physical
Measurements, 2nd editions, J.R. Taylor,
University Science Books, Sausalito, CA

3
Outline for today

Error



numerical error
data uncertainty in measurement error
Statistics & Curve Fitting





mean
standard deviation
linear regression
t-test
ANOVA
4
Types of Error

General Error (cannot blame computer) :

Blunders


Formulation or model error


incomplete mathematical model
Data uncertainty


human error
limited to significant figures in physical measurements
Numerical Error:

Round-off error (due to computer approximations)

Truncation error (due to mathematical approximations)
5

Gare Montparnasse, Paris, 1895
6
Accuracy and Precision
• Accuracy: how closely a
computed/measured value
agrees with true value
• Precision: how closely
individual
computed/measured
values agree with each
other
(a) inaccurate and imprecise
(b) accurate and imprecise
(c) inaccurate and precise
(d) accurate and precise
Note: Inaccuracy = bias
Imprecision = uncertainty
Figure 4.1, Chapra
7
Accuracy and Precision
• Inaccuracy: systemic
deviation from truth
• Imprecision: magnitude
of scatter
(a) inaccurate and imprecise
(b) accurate and imprecise
(c) inaccurate and precise
(d) accurate and precise
Note: Inaccuracy = bias
Imprecision = uncertainty
Figure 4.1, Chapra
8
Error, Accuracy and Precision
In this class we refer to Error as collective
term to represent both inaccuracy and
imprecision of our predictions
9
Round-off Errors



Occur because digital computers have a limited
ability to represent numbers
Digital computers have size & precision limits on
their ability to represent numbers
Some numerical manipulations highly sensitive
to round-off errors arising from
 mathematical
considerations and/or
 performance of arithmetic operations on computers
10
Computer Representation of Numbers

Numerical round-off errors are directly related to
way numbers are stored in computer
 The
fundamental unit whereby information is represented
is a word
A
word consists of a string of binary digits or bits
 Numbers
are stored in one or more words, e.g.,
-173 could look like this in binary on a 16-bit computer:
off “0”
on “1”
(10101101)2=27+25+23+22+20=17310
11
As good as it gets on our PCs …
-1.797693134862316 x 10308
1.797693134862316 x 10308
0
- Overflow
15 significant
figures
Underflow
15 significant
figures
Overflow +
“Hole”
on either
side of
zero
-2.225073858507201 x 10-308
2.225073858507201 x 10-308
For 64-bit, IEEE double precision format systems 12
Implications of Finite Number of bits (1)

Range
 Finite
range of numbers a computer can represent
 Overflow

error – bigger than computer can handle
For double precision (MATLAB and Excel): >1.7977 x 10308
 Underflow

error – smaller than computer can handle
For double precision (MATLAB and Excel): <2.2251 x 10-308
set format long and use realmax and realmin
in MATLAB to test your computer for range
 Can
13
Implications of Finite Number of bits (2)

Precision
 Some
numbers cannot be expressed with a finite number
of significant figures, e.g., π, e, √7
14
Round-Off Error and
Common Arithmetic Operations

Addition





Mantissa of number with smaller exponent is modified so both
are the same and decimal points are aligned
Result is chopped
Example: hypothetical 4-digit mantissa & 1-digit exponent computer
1.557 + 0.04341 = 0.1557 x 101 + 0.004341 x 101 (so they have same exponent)
= 0.160041 x 101 = 0.1600 x 101 (because of 4-digit mantissa)
Subtraction


Similar to addition, but sign of subtrahend is reversed
Severe loss of significance during subtraction of nearly equal
numbers → one of the biggest sources of round-off error in
numerical methods – subtractive cancellation
15
Round-Off Error and
Large Computations
Even though an individual round-off error could
be small, the cumulative effect over the course
of a large computation can be significant!!
 Large
numbers of computations
 Computations interdependent
 Later calculations depend on results of earlier ones
16
Particular Problems Arising from
Round-Off Error (1)

Adding a small and a large number



Common problem in summing infinite series (like the Taylor
series) where initial terms are large compared to the later terms
Mitigate by summing in the reverse order so each new term is
comparable in size to the accumulated sum (add small
numbers first)
Subtractive cancellation



Round-off error induced from subtracting two nearly equal
floating-point numbers
Example: finding roots of a quadratic equation or parabola
Mitigate by using alternative formulation of model to minimize
problem
17
Particular Problems Arising from
Round-Off Error (2)

Smearing



Occurs when individual terms in a summation are > summation
itself (positive and negative numbers in summation)
Really a form of subtractive cancellation – mitigate by using
alternative formulation of model to minimize problem
Inner Products


Common problem in solution of simultaneous linear algebraic
equations
Use double precision to mitigate problem (MATLAB does this
automatically)
n
x y
i 1
i
i
 x1 y1  x2 y2  ...  xn yn
18
Truncation Errors


Occur when exact mathematical formulations
are represented by approximations
Example:
Taylor series
19
Taylor series widely used to express
functions in an approximate fashion
Taylor’s Theorem:
Any smooth function can be approximated as a polynomial
Taylor series expansions
where h = xi+1 - xi
0th
f ( xi 1 )  f ( xi )
1st
f ( xi 1 )  f ( xi )  f ' ( xi )( h)
2nd
f " ( xi )
f ( xi 1 )  f ( xi )  f ( xi )( h) 
( h) 2
2!
3rd
f " ( xi )
f (3) ( xi )
2
f ( xi 1 )  f ( xi )  f ( xi )( h) 
( h) 
( h) 3
2!
3!
4th
f " ( xi )
f (3) ( xi )
f ( 4) ( xi )
2
3
f ( xi 1 )  f ( xi )  f ( xi )( h) 
( h) 
( h) 
( h) 4
2!
3!
4!
'
'
'
20
Each term adds more information:
e.g., f(x) = - 0.1x4 - 0.15x3 - 0.5x2 - 0.25x + 1.2 at x = 1
= 1.2
≈ 1.2 – 0.25(1) = 0.95
Figure 4.6, Chapra, p. 93
≈ 1.2 – 0.25(1) –(1.0/(1*2))*12 = 0.45
= 1.2 – 0.25(1) – (1.0/(1*2))*12
– (0.9/(1*2*3))*13 = 0.3
21
Total Numerical Error

Sum of



round-off error and
truncation error
As step size ↓,
# computations 

round-off error  (e.g.
due to subtractive cancellation
or large numbers of
computations)



truncation error ↓
Point of diminishing returns is when
round-off error begins to negate
benefits of step-size reduction
Figure 4.10, Chapra, p. 104
Trade-off here
22
Control of Numerical Errors

Experience and judgment of engineer

Practical programming guidelines:


Avoid subtracting two nearly equal numbers
Sort the numbers and work with the smallest numbers first

Use theoretical formulations to predict total numerical
errors when possible (small-scale tasks)

Check results by substituting back in original model and
see if it actually makes sense

Perform numerical experiments to increase awareness


Change step size or method to cross-check
Have two independent groups perform same calculations
23
Measurements & Uncertainty
24
Errors as Uncertainties




Error in scientific measurement means the
inevitable uncertainty that accompanies all
measurements
As such, errors are not mistakes, you cannot
eliminate them by being very careful
The best we can hope to do is to ensure that
errors are as small as reasonably possible
In this section, words error and uncertainty are
used interchangeably
25
Inevitability of Uncertainty






Carpenter wants to measure the height of
doorway before installing a door
First rough measurement: 210 cm
If pressed, the carpenter might admit that the
height in anywhere between 205 & 215 cm
For a more precise measurement, he uses a
tape measure: 211.3 cm
How can he be sure it’s not 211.3001 cm?
Use a more precise tape?
26
Measuring Length with Ruler
27
Measuring Length with Ruler
28
Measuring Length with Ruler
Note: markings are 1 mm apart
Best estimate of length = 82.5 mm
Probable range: 82 to 83 mm
We have measured the length to the nearest millimeter
29
How To Report & Use Uncertainties
Best estimate ± uncertainty
 In general, the result of any measurement
of quantity x is stated as
(measured value of x) = xbest ± Δx
 Δx is called uncertainty, or error, or margin
of error
 Δx is always positive

30
Basic Rules About Uncertainty



Δx cannot be known/stated with too much
precision; it cannot conceivably be known to 4
significant figures
Rule for stating uncertainties: Experimental
uncertainties should almost always be rounded
to one significant figure
Example: if some calculation yields Δx=0.02385,
it should be rounded to Δx=0.02
31
Basic Rules About Uncertainty


Rule for stating answers: The last significant
figure in any stated answer should usually be of
the same order of magnitude (in the same
decimal position) as the uncertainty
Examples:



The answer 92.81 with uncertainty 0.3 should be
rounded as 92.8 ± 0.3
If the uncertainty is 3, then the answer should be
rounded as 93 ± 3
If the uncertainty is 30, then the answer should be
rounded as 90 ± 30
32
Propagation Of Uncertainty
33
Statistics & Curve Fitting
34
Curve Fitting
Could plot points and
sketch a curve that
visually conforms to the
data

Three different ways
shown:
a)
Least-squares regression for
data with scatter (covered)
b)
Linear interpolation for
precise data
c)
Curvilinear interpolation for
precise data
Figure PT4.1, Chapra

Curve Fitting and Engineering Practice

Estimation of intermediate numbers from tables in
design handbooks → interpolation

Trend analysis – use pattern of data to make
predictions:
 Imprecise or “noisy” data → regression (least-squares)
 Precise data → interpolation (interpolating polynomials)

Hypothesis testing – compare existing
mathematical model with measured data
 Determine unknown model coefficient values … or …
 Compare predicted values with observed values to test
model adequacy
You’ve Got a Problem … especially if you are this guy

Wind tunnel data relating force
of air resistance (F) to wind
velocity (v) for our friend the
bungee jumper
Figure 13.1, Chapra

The data can be used to
discover the relationship and
find a drag coefficient (cd), i.e.,
 As F  , v 
How to fit the “best” line or curve
to these data?

Data is not smooth, especially
at higher v’s
 If F = 0 at v = 0, then the
relationship may not be linear
Figure 13.2, Chapra
Before We Can Discuss Regression
Techniques … We Need To Review
basic
terminology
descriptive statistics
for talking about sets of data
Data from TABLE 13.3
Basic Terminology
Maximum? 6.775
Minimum? 6.395
Range?
6.775 - 6.395 = 0.380
Individual data points, yi
y1 = 6.395
y2 = 6.435
↓
y24 = 6.775
Number of observations?
Degrees of freedom?
Residual?
yi  y
n = 24
n – 1 = 23
Use Descriptive Statistics To
Characterize Data Sets:
Location of center of distribution of the data
 Arithmetic
mean
 Median (midpoint of data, or 50th percentile)
 Mode (value that occurs most frequently)
Degree of spread of the data set
 Standard
deviation
 Variance
 Coefficient
of variation (c.v.)
Data from TABLE 13.3
Arithmetic Mean
y

y
i
n
158.4
y
 6.6
24
Data from TABLE 13.3
Standard Deviation
St
sy 

n 1
2
(
y

y
)
 i
n 1
St: total sum of squares of residuals
between data points and mean
0.217000
sy 
 0.097133
24  1
Data from TABLE 13.3
Variance
s y2 

2
(
y

y
)
 i
St

n 1
n 1
2
2
y

(
y
)
 i  i /n
n 1
0.217000
s 
 0.009435
24  1
2
y
Coefficient of
Variation (c.v.)
c.v. = standard deviation / mean
Normalized measure of spread
c.v. 
sy
y
100%
0.097133
c.v. 
100%  1.47%
6.6
Data from TABLE 13.3
Data from TABLE 13.3
Histogram of data
Figure 12.4, Chapra
For a large set of data,
histogram can be
approximated by
a smooth, symmetric
bell-shaped curve
→ normal distribution
Confidence Intervals

If a data set is normally distributed, ~68% of the total
measurements will fall within the range defined by
y  s y to y  s y

Similarly, ~95% of the total measurements will be
encompassed by the range
y  2s y to y  2s y
Descriptive Statistics in MATLAB
>>% s holds data from Table 13.2
>>s=[6.395;6.435;6.485;…;6.775]
>>mean(s), median(s),
>>min(s),
mode(s)
max(s)
ans = 6.6
ans = 6.395
ans = 6.61
ans = 6.775
ans = 6.555
>>var(s),
>>range=max(s)std(s)
min(s)
ans =
range = 0.38
0.0094348
ans = 0.097133 >>[n,x]=hist(s)
n =
1
1
3
1
4
3
5
2
2
2
x = 6.414 6.452 6.49 6.528 6.566 6.604 6.642 6.68 6.718 6.756
n is the number of elements in each bin; x is a vector specifying the midpoint of each bin
Back to the Bungee Jumper Wind Tunnel
Data … is the mean a good fit to the data?
Figure 12.1, Chapra
velocity, m/s
10
20
30
40
50
60
70
80
mean:
Force, N
25
70
380
550
610
1220
830
1450
642
Figure 13.8a, Chapra
not very!!! distribution
of residuals is large
Figure 13.2, Chapra
Curve Fitting Techniques

Least-squaresFigure
regression
12.8, Chapra, p. 209
Figure 13.8b, Chapra
 Linear
 Polynomial
 General
linear least-squares
 Nonlinear

Interpolation (not covered)
 Polynomial
 Splines
Can reduce the distribution
of the residuals if use curvefitting techniques such as
linear least-squares regression
Linear Least-Squares Regression

Linear least-squares regression, the simplest
example of a least-squares approximation is fitting
a straight line to a set of paired observations:
(x1, y1), (x2, y2), …, (xn, yn)

Mathematical expression for a straight line:
y = a0+ a1x + e
error or residual
intercept
slope
Least-Squares Regression:
Important Statistical Assumptions

Each x has a fixed value; it is not random and it
is known without error, this means that
 x values must be error-free
 regression of y versus x is not
the same as the
regression of x versus y

The y values are independent random variables
and all have the same variance

The y values for a given x must be normally
distributed
Residuals in Linear Regression

Regression line is a
measure of central
tendency for paired
observations (i.e.,
data points)

Residuals (ei) in
linear regression
represent the
vertical distance
between a data
point and the
regression line
Figure 13.7, Chapra
How to Get the “Best” Fit:
Minimize the sum of the squares of the residuals
between the measured y and the y calculated with the (linear)
model:
n
S r   ei2
i 1
n
  ( yi ,measured  yi ,model ) 2
i 1
n
  ( yi  a0  a1 xi ) 2
i 1
Yields a unique line for a given dataset
How do we compute the best a0 and a1?
n
S r   ( yi  a0  a1 xi ) 2
i 1
One way is to use optimization techniques since
looking for a minimum (more common for nonlinear
case)… or …
Another way is to solve the normal equations for a0
and a1 according to the derivation in the next few
slides
Derivation of Normal Equations Used to
Solve for a0 and a1
n
S r   ( yi  a0  a1 xi ) 2
i 1

First, differentiate the sum of
the squares of the residuals
with respect to each unknown
coefficient
¶Sr
= -2å (yi - a0 - a1 xi )
¶a0
¶Sr
= -2å[(yi - a0 - a1 xi )]xi
¶a1
Derivation of Normal Equations Used to
Solve for a0 and a1 - continued

Set derivatives = 0

Will result in a
minimum Sr

Can be expressed as
S r
 2 ( yi  a0  a1 xi )
a0
0
S r
 2 [( yi  a0  a1 xi )] xi  0
a1
0   yi   a0   a1 xi
0   yi xi   a0 xi   a1 xi2
Derivation of Normal Equations Used to
Solve for a0 and a1 - continued

Realizing ∑a0 = na0, we can express these
equations as a set of two simultaneous linear
equations with two unknowns (a0 and a1) called
the normal equations:
Normal Equations:
0   yi   a0   a1 xi
na0   xi a1   yi
0   yi xi   a0 xi   a1 xi2




  xi a0    xi2 a1   xi yi








Derivation of Normal Equations Used to
Solve for a0 and a1 - continued
na0   xi a1   yi
 n

 xi




  xi a0    xi2 a1   xi yi









Finally, can solve
these normal
equations for a0
and a1
a1 
 x  a     y 
 x   a   x y 
i
2
i
i
0
i
1
n xi yi   xi  yi



n x    xi 


2
i
2
and
i
a0  y  a1 x
Improvement Due to Linear Regression
Figure 12.8, Chapra, p. 209
mean of the dependent
variable
sy/x
sy
best-fit line
Figure 13.8, Chapra
Spread of data around the mean
of the dependent variable
Spread of data around the best-fit line
The reduction in spread in going from (a) to (b), indicated by bellshape curves, represents improvement due to linear regression
Improvement Due to Linear Regression
Figure 12.8, Chapra, p. 209
sy/x
mean of the dependent
variable
Total sum of squares around the
mean of dependent variable
n
S t   ( yi  y )
i 1
2
sy
best-fit line
Sum of squares of residuals
around the best-fit
regression line
n
S r   ( yi  a0  a1 xi )
2
i 1
Coefficient of determination quantifies improvement or error
reduction due to describing data in terms of a straight line rather
than an average value
St  S r
r 
St
2
“Goodness” of Fit


St - Sr quantifies error reduction
due to using line instead of
mean
Normalize by St because scale
sensitive → r 2 = coefficient of
determination
small residual errors
r2 → 1
St  S r
r 
St
2



Used for comparison of several
regressions
Value of zero represents no
improvement
Value of 1 is a perfect fit, the line
explains 100% of data variability
large residual errors
r2 << 1
Figure 12.9, Chapra
Linearization of Nonlinear
Relationships

What to do when relationship
is nonlinear?

One option is polynomial
regression
Another option is to linearize
the data using transformation
techniques




Exponential
Power
Saturation-growth-rate
Figure 14.1, Chapra
Linearization Transformation Examples

Exponential model:
y  1e
1 x
Figure 13.11, Chapra

Used to characterize quantities
that increase (+β1) or decrease
(-β1) at a rate directly proportional
to their own magnitude, e.g.,
population growth or radioactive
decay

Take ln of both sides to linearize
data:
ln y  ln 1  1x
Linearization Transformation Examples

Power model:
y  2 x
2
Figure 13.11, Chapra

Widely applicable in all fields
of engineering

Take log of both sides to
linearize data:
log y  2 log x  log  2
Linearization Transformation Examples

Saturation-growth-rate model
x
y  3
3  x
Figure 13.11, Chapra

Used to characterize
population growth under
limiting conditions or enzyme
kinetics

To linearize, invert equation to
give:
1 3  1  1
  
y 3  x  3
Linear Regression with MATLAB

Polyfit can be used to determine the slope
and y-intercept as follows:
>>x=[10 20 30 40 50 60 70 80];
>>y=[25 70 380 550 610 1220 830 1450];
>>a=polyfit(x,y,1)
use 1 for linear (1st order)
a = 19.4702 -234.2857

Polyval can be used to compute a value using
the coefficients as follows:
>>y = polyval(a,45)
y = 641.8750
Polynomial Regression

Extend the linear leastsquares procedure to fit
data to a higher order
polynomial

For a quadratic (2nd order
polynomial), will have a
system of 3 normal
equations to solve
instead of 2 as for linear

For higher mth-order
polynomials, will have a
system of m+1 normal
equations to solve
Data not suited for linear least-squares
regression
Figure 14.1, Chapra
Nonlinear Regression (not covered)

Cases in engineering where nonlinear models –
models that have a nonlinear dependence on
their parameters – must be fit to data

For example,
f ( x)  a0 (1  e  a1x )  eerror

Like linear models in that we still minimize the
sum of the squares of the residuals

Most convenient to do this with optimization
More Statistics: Comparing 2 Means

depends on mean and
amount of variability

can tell there is a
difference when
variability is low
use t-test to do this
mathematically

http://www.socialresearchmethods.net/kb/stat_t.php
69
The t-test Is A Ratio Of “Signal To Noise”
Remember, variance (var) is just the
square of the standard deviation
Standard Error of difference between means
http://www.socialresearchmethods.net/kb/stat_t.php
70
How It Works ... Once You Have A t-value

Need to set a risk level (called the alpha level) – a
typical value is 0.05 which means that five times out of
a hundred you would find a statistically significant
difference between the means even if there was none
(i.e., by "chance").

Degrees of freedom (df) for the test = sum of # data
points in both groups (N) minus 2.

Given the alpha level, the df, and the t-value, you can
use a t-test table or computer program to determine
whether the t-value is large enough to be significant.

If calculated t is larger than t (alpha, N-2) in table, you can
conclude that the difference between the means for
the two groups is different (even given the variability).
http://www.socialresearchmethods.net/kb/stat_t.php
71
What About More Than 2 Sets Of Data?

ANOVA = Analysis of Variance commonly used for more
than 2, if ...




k samples are random and independently selected
treatment responses are normally distributed
treatment responses have equal variances
ANOVA compares variation between groups of data
to variation within groups, i.e.,
variation between groups
F =
variation within groups
Source: http://www.uwlax.edu/faculty/baggett/Math_145/HANDOUTS/anova.pdf
72
Steps for ANOVA


Define k population or treatment means being compared in context
of problem
Set up hypotheses to be tested, i.e.,

H0: all means are equal
 Ha: not all means are equal (no claim as to which ones not)



Choose risk level, alpha (=0.05 is a typical level)
Check if assumptions reasonable (previous slide)
Calculate the test statistic ... pretty involved ...see next page!
Note: usually use a computer program, but is helpful to know what
computer is doing ... can do simple problems by hand
Source: http://www.uwlax.edu/faculty/baggett/Math_145/HANDOUTS/anova.pdf
73
ANOVA calculations

Collect this info from data set:
Source: http://www.uwlax.edu/faculty/baggett/Math_145/HANDOUTS/anova.pdf
74
ANOVA calculations

Fill out a table like this to compute the F ratio
statistic:
Source: http://www.uwlax.edu/faculty/baggett/Math_145/HANDOUTS/anova.pdf
75
ANOVA calculations



Now what do we do with F statistic ?
Compare it to an F distribution like we did with the t-test
This time we need




to look up F (1- alpha)(k-1, N-k)
This time we need to compare


alpha
df of numerator (k-1)
df of denominator (N-k)
F ≥ F (1- alpha)(k-1, N-k)
If yes, then more evidence against H0, reject H0
Source: http://www.uwlax.edu/faculty/baggett/Math_145/HANDOUTS/anova.pdf
76
The End
77