Survey

Survey

Document related concepts

Transcript

Statistical & Uncertainty Analysis UC Summer REU Programs Instructor: Lilit Yeghiazarian, PhD Environmental Engineering Program 1 Instructor Dr. Lilit Yeghiazarian Environmental Engineering Program Office: 746 Engineering Research Center (ERC) Email: [email protected] Phone: 513-556-3623 2 Textbooks Applied Numerical Methods with MATLAB for Engineers and Scientists, 3rd edition, S.C. Chapra, McGraw-Hill Companies, Inc., 2012 An Introduction to Error Analysis: The Study of Uncertainties in Physical Measurements, 2nd editions, J.R. Taylor, University Science Books, Sausalito, CA 3 Outline for today Error numerical error data uncertainty in measurement error Statistics & Curve Fitting mean standard deviation linear regression t-test ANOVA 4 Types of Error General Error (cannot blame computer) : Blunders Formulation or model error incomplete mathematical model Data uncertainty human error limited to significant figures in physical measurements Numerical Error: Round-off error (due to computer approximations) Truncation error (due to mathematical approximations) 5 Gare Montparnasse, Paris, 1895 6 Accuracy and Precision • Accuracy: how closely a computed/measured value agrees with true value • Precision: how closely individual computed/measured values agree with each other (a) inaccurate and imprecise (b) accurate and imprecise (c) inaccurate and precise (d) accurate and precise Note: Inaccuracy = bias Imprecision = uncertainty Figure 4.1, Chapra 7 Accuracy and Precision • Inaccuracy: systemic deviation from truth • Imprecision: magnitude of scatter (a) inaccurate and imprecise (b) accurate and imprecise (c) inaccurate and precise (d) accurate and precise Note: Inaccuracy = bias Imprecision = uncertainty Figure 4.1, Chapra 8 Error, Accuracy and Precision In this class we refer to Error as collective term to represent both inaccuracy and imprecision of our predictions 9 Round-off Errors Occur because digital computers have a limited ability to represent numbers Digital computers have size & precision limits on their ability to represent numbers Some numerical manipulations highly sensitive to round-off errors arising from mathematical considerations and/or performance of arithmetic operations on computers 10 Computer Representation of Numbers Numerical round-off errors are directly related to way numbers are stored in computer The fundamental unit whereby information is represented is a word A word consists of a string of binary digits or bits Numbers are stored in one or more words, e.g., -173 could look like this in binary on a 16-bit computer: off “0” on “1” (10101101)2=27+25+23+22+20=17310 11 As good as it gets on our PCs … -1.797693134862316 x 10308 1.797693134862316 x 10308 0 - Overflow 15 significant figures Underflow 15 significant figures Overflow + “Hole” on either side of zero -2.225073858507201 x 10-308 2.225073858507201 x 10-308 For 64-bit, IEEE double precision format systems 12 Implications of Finite Number of bits (1) Range Finite range of numbers a computer can represent Overflow error – bigger than computer can handle For double precision (MATLAB and Excel): >1.7977 x 10308 Underflow error – smaller than computer can handle For double precision (MATLAB and Excel): <2.2251 x 10-308 set format long and use realmax and realmin in MATLAB to test your computer for range Can 13 Implications of Finite Number of bits (2) Precision Some numbers cannot be expressed with a finite number of significant figures, e.g., π, e, √7 14 Round-Off Error and Common Arithmetic Operations Addition Mantissa of number with smaller exponent is modified so both are the same and decimal points are aligned Result is chopped Example: hypothetical 4-digit mantissa & 1-digit exponent computer 1.557 + 0.04341 = 0.1557 x 101 + 0.004341 x 101 (so they have same exponent) = 0.160041 x 101 = 0.1600 x 101 (because of 4-digit mantissa) Subtraction Similar to addition, but sign of subtrahend is reversed Severe loss of significance during subtraction of nearly equal numbers → one of the biggest sources of round-off error in numerical methods – subtractive cancellation 15 Round-Off Error and Large Computations Even though an individual round-off error could be small, the cumulative effect over the course of a large computation can be significant!! Large numbers of computations Computations interdependent Later calculations depend on results of earlier ones 16 Particular Problems Arising from Round-Off Error (1) Adding a small and a large number Common problem in summing infinite series (like the Taylor series) where initial terms are large compared to the later terms Mitigate by summing in the reverse order so each new term is comparable in size to the accumulated sum (add small numbers first) Subtractive cancellation Round-off error induced from subtracting two nearly equal floating-point numbers Example: finding roots of a quadratic equation or parabola Mitigate by using alternative formulation of model to minimize problem 17 Particular Problems Arising from Round-Off Error (2) Smearing Occurs when individual terms in a summation are > summation itself (positive and negative numbers in summation) Really a form of subtractive cancellation – mitigate by using alternative formulation of model to minimize problem Inner Products Common problem in solution of simultaneous linear algebraic equations Use double precision to mitigate problem (MATLAB does this automatically) n x y i 1 i i x1 y1 x2 y2 ... xn yn 18 Truncation Errors Occur when exact mathematical formulations are represented by approximations Example: Taylor series 19 Taylor series widely used to express functions in an approximate fashion Taylor’s Theorem: Any smooth function can be approximated as a polynomial Taylor series expansions where h = xi+1 - xi 0th f ( xi 1 ) f ( xi ) 1st f ( xi 1 ) f ( xi ) f ' ( xi )( h) 2nd f " ( xi ) f ( xi 1 ) f ( xi ) f ( xi )( h) ( h) 2 2! 3rd f " ( xi ) f (3) ( xi ) 2 f ( xi 1 ) f ( xi ) f ( xi )( h) ( h) ( h) 3 2! 3! 4th f " ( xi ) f (3) ( xi ) f ( 4) ( xi ) 2 3 f ( xi 1 ) f ( xi ) f ( xi )( h) ( h) ( h) ( h) 4 2! 3! 4! ' ' ' 20 Each term adds more information: e.g., f(x) = - 0.1x4 - 0.15x3 - 0.5x2 - 0.25x + 1.2 at x = 1 = 1.2 ≈ 1.2 – 0.25(1) = 0.95 Figure 4.6, Chapra, p. 93 ≈ 1.2 – 0.25(1) –(1.0/(1*2))*12 = 0.45 = 1.2 – 0.25(1) – (1.0/(1*2))*12 – (0.9/(1*2*3))*13 = 0.3 21 Total Numerical Error Sum of round-off error and truncation error As step size ↓, # computations round-off error (e.g. due to subtractive cancellation or large numbers of computations) truncation error ↓ Point of diminishing returns is when round-off error begins to negate benefits of step-size reduction Figure 4.10, Chapra, p. 104 Trade-off here 22 Control of Numerical Errors Experience and judgment of engineer Practical programming guidelines: Avoid subtracting two nearly equal numbers Sort the numbers and work with the smallest numbers first Use theoretical formulations to predict total numerical errors when possible (small-scale tasks) Check results by substituting back in original model and see if it actually makes sense Perform numerical experiments to increase awareness Change step size or method to cross-check Have two independent groups perform same calculations 23 Measurements & Uncertainty 24 Errors as Uncertainties Error in scientific measurement means the inevitable uncertainty that accompanies all measurements As such, errors are not mistakes, you cannot eliminate them by being very careful The best we can hope to do is to ensure that errors are as small as reasonably possible In this section, words error and uncertainty are used interchangeably 25 Inevitability of Uncertainty Carpenter wants to measure the height of doorway before installing a door First rough measurement: 210 cm If pressed, the carpenter might admit that the height in anywhere between 205 & 215 cm For a more precise measurement, he uses a tape measure: 211.3 cm How can he be sure it’s not 211.3001 cm? Use a more precise tape? 26 Measuring Length with Ruler 27 Measuring Length with Ruler 28 Measuring Length with Ruler Note: markings are 1 mm apart Best estimate of length = 82.5 mm Probable range: 82 to 83 mm We have measured the length to the nearest millimeter 29 How To Report & Use Uncertainties Best estimate ± uncertainty In general, the result of any measurement of quantity x is stated as (measured value of x) = xbest ± Δx Δx is called uncertainty, or error, or margin of error Δx is always positive 30 Basic Rules About Uncertainty Δx cannot be known/stated with too much precision; it cannot conceivably be known to 4 significant figures Rule for stating uncertainties: Experimental uncertainties should almost always be rounded to one significant figure Example: if some calculation yields Δx=0.02385, it should be rounded to Δx=0.02 31 Basic Rules About Uncertainty Rule for stating answers: The last significant figure in any stated answer should usually be of the same order of magnitude (in the same decimal position) as the uncertainty Examples: The answer 92.81 with uncertainty 0.3 should be rounded as 92.8 ± 0.3 If the uncertainty is 3, then the answer should be rounded as 93 ± 3 If the uncertainty is 30, then the answer should be rounded as 90 ± 30 32 Propagation Of Uncertainty 33 Statistics & Curve Fitting 34 Curve Fitting Could plot points and sketch a curve that visually conforms to the data Three different ways shown: a) Least-squares regression for data with scatter (covered) b) Linear interpolation for precise data c) Curvilinear interpolation for precise data Figure PT4.1, Chapra Curve Fitting and Engineering Practice Estimation of intermediate numbers from tables in design handbooks → interpolation Trend analysis – use pattern of data to make predictions: Imprecise or “noisy” data → regression (least-squares) Precise data → interpolation (interpolating polynomials) Hypothesis testing – compare existing mathematical model with measured data Determine unknown model coefficient values … or … Compare predicted values with observed values to test model adequacy You’ve Got a Problem … especially if you are this guy Wind tunnel data relating force of air resistance (F) to wind velocity (v) for our friend the bungee jumper Figure 13.1, Chapra The data can be used to discover the relationship and find a drag coefficient (cd), i.e., As F , v How to fit the “best” line or curve to these data? Data is not smooth, especially at higher v’s If F = 0 at v = 0, then the relationship may not be linear Figure 13.2, Chapra Before We Can Discuss Regression Techniques … We Need To Review basic terminology descriptive statistics for talking about sets of data Data from TABLE 13.3 Basic Terminology Maximum? 6.775 Minimum? 6.395 Range? 6.775 - 6.395 = 0.380 Individual data points, yi y1 = 6.395 y2 = 6.435 ↓ y24 = 6.775 Number of observations? Degrees of freedom? Residual? yi y n = 24 n – 1 = 23 Use Descriptive Statistics To Characterize Data Sets: Location of center of distribution of the data Arithmetic mean Median (midpoint of data, or 50th percentile) Mode (value that occurs most frequently) Degree of spread of the data set Standard deviation Variance Coefficient of variation (c.v.) Data from TABLE 13.3 Arithmetic Mean y y i n 158.4 y 6.6 24 Data from TABLE 13.3 Standard Deviation St sy n 1 2 ( y y ) i n 1 St: total sum of squares of residuals between data points and mean 0.217000 sy 0.097133 24 1 Data from TABLE 13.3 Variance s y2 2 ( y y ) i St n 1 n 1 2 2 y ( y ) i i /n n 1 0.217000 s 0.009435 24 1 2 y Coefficient of Variation (c.v.) c.v. = standard deviation / mean Normalized measure of spread c.v. sy y 100% 0.097133 c.v. 100% 1.47% 6.6 Data from TABLE 13.3 Data from TABLE 13.3 Histogram of data Figure 12.4, Chapra For a large set of data, histogram can be approximated by a smooth, symmetric bell-shaped curve → normal distribution Confidence Intervals If a data set is normally distributed, ~68% of the total measurements will fall within the range defined by y s y to y s y Similarly, ~95% of the total measurements will be encompassed by the range y 2s y to y 2s y Descriptive Statistics in MATLAB >>% s holds data from Table 13.2 >>s=[6.395;6.435;6.485;…;6.775] >>mean(s), median(s), >>min(s), mode(s) max(s) ans = 6.6 ans = 6.395 ans = 6.61 ans = 6.775 ans = 6.555 >>var(s), >>range=max(s)std(s) min(s) ans = range = 0.38 0.0094348 ans = 0.097133 >>[n,x]=hist(s) n = 1 1 3 1 4 3 5 2 2 2 x = 6.414 6.452 6.49 6.528 6.566 6.604 6.642 6.68 6.718 6.756 n is the number of elements in each bin; x is a vector specifying the midpoint of each bin Back to the Bungee Jumper Wind Tunnel Data … is the mean a good fit to the data? Figure 12.1, Chapra velocity, m/s 10 20 30 40 50 60 70 80 mean: Force, N 25 70 380 550 610 1220 830 1450 642 Figure 13.8a, Chapra not very!!! distribution of residuals is large Figure 13.2, Chapra Curve Fitting Techniques Least-squaresFigure regression 12.8, Chapra, p. 209 Figure 13.8b, Chapra Linear Polynomial General linear least-squares Nonlinear Interpolation (not covered) Polynomial Splines Can reduce the distribution of the residuals if use curvefitting techniques such as linear least-squares regression Linear Least-Squares Regression Linear least-squares regression, the simplest example of a least-squares approximation is fitting a straight line to a set of paired observations: (x1, y1), (x2, y2), …, (xn, yn) Mathematical expression for a straight line: y = a0+ a1x + e error or residual intercept slope Least-Squares Regression: Important Statistical Assumptions Each x has a fixed value; it is not random and it is known without error, this means that x values must be error-free regression of y versus x is not the same as the regression of x versus y The y values are independent random variables and all have the same variance The y values for a given x must be normally distributed Residuals in Linear Regression Regression line is a measure of central tendency for paired observations (i.e., data points) Residuals (ei) in linear regression represent the vertical distance between a data point and the regression line Figure 13.7, Chapra How to Get the “Best” Fit: Minimize the sum of the squares of the residuals between the measured y and the y calculated with the (linear) model: n S r ei2 i 1 n ( yi ,measured yi ,model ) 2 i 1 n ( yi a0 a1 xi ) 2 i 1 Yields a unique line for a given dataset How do we compute the best a0 and a1? n S r ( yi a0 a1 xi ) 2 i 1 One way is to use optimization techniques since looking for a minimum (more common for nonlinear case)… or … Another way is to solve the normal equations for a0 and a1 according to the derivation in the next few slides Derivation of Normal Equations Used to Solve for a0 and a1 n S r ( yi a0 a1 xi ) 2 i 1 First, differentiate the sum of the squares of the residuals with respect to each unknown coefficient ¶Sr = -2å (yi - a0 - a1 xi ) ¶a0 ¶Sr = -2å[(yi - a0 - a1 xi )]xi ¶a1 Derivation of Normal Equations Used to Solve for a0 and a1 - continued Set derivatives = 0 Will result in a minimum Sr Can be expressed as S r 2 ( yi a0 a1 xi ) a0 0 S r 2 [( yi a0 a1 xi )] xi 0 a1 0 yi a0 a1 xi 0 yi xi a0 xi a1 xi2 Derivation of Normal Equations Used to Solve for a0 and a1 - continued Realizing ∑a0 = na0, we can express these equations as a set of two simultaneous linear equations with two unknowns (a0 and a1) called the normal equations: Normal Equations: 0 yi a0 a1 xi na0 xi a1 yi 0 yi xi a0 xi a1 xi2 xi a0 xi2 a1 xi yi Derivation of Normal Equations Used to Solve for a0 and a1 - continued na0 xi a1 yi n xi xi a0 xi2 a1 xi yi Finally, can solve these normal equations for a0 and a1 a1 x a y x a x y i 2 i i 0 i 1 n xi yi xi yi n x xi 2 i 2 and i a0 y a1 x Improvement Due to Linear Regression Figure 12.8, Chapra, p. 209 mean of the dependent variable sy/x sy best-fit line Figure 13.8, Chapra Spread of data around the mean of the dependent variable Spread of data around the best-fit line The reduction in spread in going from (a) to (b), indicated by bellshape curves, represents improvement due to linear regression Improvement Due to Linear Regression Figure 12.8, Chapra, p. 209 sy/x mean of the dependent variable Total sum of squares around the mean of dependent variable n S t ( yi y ) i 1 2 sy best-fit line Sum of squares of residuals around the best-fit regression line n S r ( yi a0 a1 xi ) 2 i 1 Coefficient of determination quantifies improvement or error reduction due to describing data in terms of a straight line rather than an average value St S r r St 2 “Goodness” of Fit St - Sr quantifies error reduction due to using line instead of mean Normalize by St because scale sensitive → r 2 = coefficient of determination small residual errors r2 → 1 St S r r St 2 Used for comparison of several regressions Value of zero represents no improvement Value of 1 is a perfect fit, the line explains 100% of data variability large residual errors r2 << 1 Figure 12.9, Chapra Linearization of Nonlinear Relationships What to do when relationship is nonlinear? One option is polynomial regression Another option is to linearize the data using transformation techniques Exponential Power Saturation-growth-rate Figure 14.1, Chapra Linearization Transformation Examples Exponential model: y 1e 1 x Figure 13.11, Chapra Used to characterize quantities that increase (+β1) or decrease (-β1) at a rate directly proportional to their own magnitude, e.g., population growth or radioactive decay Take ln of both sides to linearize data: ln y ln 1 1x Linearization Transformation Examples Power model: y 2 x 2 Figure 13.11, Chapra Widely applicable in all fields of engineering Take log of both sides to linearize data: log y 2 log x log 2 Linearization Transformation Examples Saturation-growth-rate model x y 3 3 x Figure 13.11, Chapra Used to characterize population growth under limiting conditions or enzyme kinetics To linearize, invert equation to give: 1 3 1 1 y 3 x 3 Linear Regression with MATLAB Polyfit can be used to determine the slope and y-intercept as follows: >>x=[10 20 30 40 50 60 70 80]; >>y=[25 70 380 550 610 1220 830 1450]; >>a=polyfit(x,y,1) use 1 for linear (1st order) a = 19.4702 -234.2857 Polyval can be used to compute a value using the coefficients as follows: >>y = polyval(a,45) y = 641.8750 Polynomial Regression Extend the linear leastsquares procedure to fit data to a higher order polynomial For a quadratic (2nd order polynomial), will have a system of 3 normal equations to solve instead of 2 as for linear For higher mth-order polynomials, will have a system of m+1 normal equations to solve Data not suited for linear least-squares regression Figure 14.1, Chapra Nonlinear Regression (not covered) Cases in engineering where nonlinear models – models that have a nonlinear dependence on their parameters – must be fit to data For example, f ( x) a0 (1 e a1x ) eerror Like linear models in that we still minimize the sum of the squares of the residuals Most convenient to do this with optimization More Statistics: Comparing 2 Means depends on mean and amount of variability can tell there is a difference when variability is low use t-test to do this mathematically http://www.socialresearchmethods.net/kb/stat_t.php 69 The t-test Is A Ratio Of “Signal To Noise” Remember, variance (var) is just the square of the standard deviation Standard Error of difference between means http://www.socialresearchmethods.net/kb/stat_t.php 70 How It Works ... Once You Have A t-value Need to set a risk level (called the alpha level) – a typical value is 0.05 which means that five times out of a hundred you would find a statistically significant difference between the means even if there was none (i.e., by "chance"). Degrees of freedom (df) for the test = sum of # data points in both groups (N) minus 2. Given the alpha level, the df, and the t-value, you can use a t-test table or computer program to determine whether the t-value is large enough to be significant. If calculated t is larger than t (alpha, N-2) in table, you can conclude that the difference between the means for the two groups is different (even given the variability). http://www.socialresearchmethods.net/kb/stat_t.php 71 What About More Than 2 Sets Of Data? ANOVA = Analysis of Variance commonly used for more than 2, if ... k samples are random and independently selected treatment responses are normally distributed treatment responses have equal variances ANOVA compares variation between groups of data to variation within groups, i.e., variation between groups F = variation within groups Source: http://www.uwlax.edu/faculty/baggett/Math_145/HANDOUTS/anova.pdf 72 Steps for ANOVA Define k population or treatment means being compared in context of problem Set up hypotheses to be tested, i.e., H0: all means are equal Ha: not all means are equal (no claim as to which ones not) Choose risk level, alpha (=0.05 is a typical level) Check if assumptions reasonable (previous slide) Calculate the test statistic ... pretty involved ...see next page! Note: usually use a computer program, but is helpful to know what computer is doing ... can do simple problems by hand Source: http://www.uwlax.edu/faculty/baggett/Math_145/HANDOUTS/anova.pdf 73 ANOVA calculations Collect this info from data set: Source: http://www.uwlax.edu/faculty/baggett/Math_145/HANDOUTS/anova.pdf 74 ANOVA calculations Fill out a table like this to compute the F ratio statistic: Source: http://www.uwlax.edu/faculty/baggett/Math_145/HANDOUTS/anova.pdf 75 ANOVA calculations Now what do we do with F statistic ? Compare it to an F distribution like we did with the t-test This time we need to look up F (1- alpha)(k-1, N-k) This time we need to compare alpha df of numerator (k-1) df of denominator (N-k) F ≥ F (1- alpha)(k-1, N-k) If yes, then more evidence against H0, reject H0 Source: http://www.uwlax.edu/faculty/baggett/Math_145/HANDOUTS/anova.pdf 76 The End 77