Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Experimental Errors and Uncertainty: An Introduction Prepared for students in AE 3051 by J. M. Seitzman adapted from material made available by J. Craig JMS090800-1 AE3051 The Measurement System • Measurements – Direct/Indirect comparison (rulers, balance scale, interferometer – Calibrated system (odometer, spring scale, pressure gage) • Measurement System Physical Measurand DetectorSensor Transducer Signal Conditioner Indicator, Recorder, Controller, Computer, etc. • Issues: – Detector/Sensor: device which detects and responds to measurand – Transducer: converts measurand to an analog more easily measured (force-displacement-resistance-voltage) – Signal Cond.: amplify, filter, integrate, differentiate, convert freq. to voltage, etc. – Computer: widely used today JMS090800-2 AE3051 Example of Measurement System readout readout transducer detector detector transducer Tire Pressure Gage • • Bourdon Type Pressure Gage These are simple mechanical systems Issues – – – – JMS090800-3 mechanical vs electrical output analog vs digital calibration accuracy, precision, resolution, sensitivity, linearity, drift, backlash? AE3051 Computer Readout Systems Multichannel Data Acquisition (sequential sampling) Volt sample Chan #1 Voltages MUX S/H ADC digital words to computer Chan #2 Computer Control • • Each channel is read in sequence (10 ms to 1 ms per reading). ADC may output a reading that is from 8 to 16 bits in size. skew Readings are not recorded simultaneously – 8 bits & bipolar range yields readings from –128 to +127 (28=256) corresponding to –Full Scale and + Full Scale – 16 bits & bipolar range yields reading from –32768 to +32767 (216=65,536) so resolution is about 4½ digits or about 0.01% of Full Scale (not of reading) – With ±1v range, an 8 bit ADC has a 7.9 mv resoultion (=1/127) – With ±10v range, a 16 bit ADC has a 0.31 mv resolution (=10/32767) Key: MUX = multiplexer (switch) S/H = sample & hold (hold voltage while ADC reads) ADC = analog to digital converter (voltmeter) JMS090800-4 t AE3051 Experimental Error • Error: all measurements have some uncertainty error = ε = xmeas − xexact • Objectives 1. Minimize error so that uncertainty, u, is: -u ≤ ε ≤ +u at N:1 certainty (a statistical confidence) or xmeas − u ≤ xexact ≤ xmeas + u 2. Estimate error to determine reliability, meaningfulness of data ∆ uexact JMS090800-5 ε umeas AE3051 Accuracy and Precision • Accuracy: also called systematic or bias error – denotes something repeatably “wrong” with the measurement or experiment • Precision: also called random error or noise – denotes errors that change randomly each time you try to repeat experiment Good Accuracy Good Precision JMS090800-6 Good Precision Poor Accuracy (can calibrate) Good Accuracy Poor Precision (can average) Poor Accuracy Poor Precision AE3051 Other Related Terms • Resolution – Smallest increment of change in a system or property that a measurement device can reliably capture • Sensitivity – Change in a measurement device’s output for a unit change in the measured (input) quantity, e.g., volts/Torr for the Baratron • Dynamic Range – Maximum output of a measurement device divided by its resolution JMS090800-7 AE3051 Accuracy/Systematic Errors • Sources – Measuring system errors • difference between model of measuring system and reality • could be corrected, e.g., with better model of measurement – Measured system “errors” • influence of uncontrolled or unaccounted for variables in the experiment • the measured data may be “correct”, but may lead to an incorrect model of the object/process being studied JMS090800-8 AE3051 Some Systematic Errors of Measuring Systems •u=u(x) backlash u Model u u(x=const) Actual Actual Model Model Actual hysteresis x Nonzero offset - Background x Backlash & Hysteresis u Systematic errors can be eliminated/removed if they are known Model time Drift (e.g., offset changing in systematic way with time) u Model Actual Nonlinearity x JMS090800-9 Quantization Error (digitized data – impacts resolution) Actual x AE3051 Some Random Errors of Measuring Systems u u u(x=const) Actual Model Model time Background “Noise” (offset changing randomly with time) 0 Actual x x Detector “Noise” (random change in sensitivity of device) Disturbances – “Noise” (e.g., pickup electrical signals from other sources After data acquired, nearly impossible to separate random error (noise) sources JMS090800-10 AE3051 Uncertainty: Statistical Approaches • Probability & statistics provide a way to deal with uncertainty – we will cover only a VERY limited introduction precision (random) error Bias (systematic) error total error Frequency Bias error: systematic errors that could be removed by proper calibration theoretical distribution number of times reading falls in this range of x Precision error: random error - not directly controllable xm Xexact Bias: calibration consistent human error background JMS090800-11 Xavg Xm Precision: disturbances noise variable conditions Blunders: human error software! Either: backlash, hysteresis, friction damping drift variations in test procedure AE3051 Basic Concepts in Probability • Sample Space: all possible events or outcomes of an experiment; also can be referred to as a population. • Event: subset of sample space • Sample: finite number taken from population (e.g., 15 turbine blades taken from 8600 produced for XX-300 engine) – sample must be randomly selected from population – sample may or may not be returned to population before resampling • Probability: likelihood of an event (measured as % of successful events in sample space or population). We can often analytically compute this likelihood based on mathematics applied to the population. 0 ≤ P(event) ≤ 1. • There are many rules for computing probabilities for different kinds of events; these are the subject of courses and books on probability theory. JMS090800-12 AE3051 Numbers on one die 1 2 • 3 4 5 Number 6 Sum of numbers on 2 dice 6 1 2 3 4 5 6 7 8 9 10 11 12 Number (sum of dice) Heads in 100 coin tosses Number of Events 1 Number of Events Number of Events Examples 0 50 Number 100 These happen to be binomial distributions (repeat experiment N times; each trial is independent of others; each trial is successful or not; probability of success, p, is same for each trial). p x ( 1 − p ) N − x P( x successes in N trials ) = N x N! N = x x! ( N − x )! • JMS090800-13 These are discrete distributions but there are also continuous distributions that are defined by CDF and PDF curves (next) AE3051 Properties of Probability Distribution Functions 1.0 PDF 0.6 CDF Area = 0.6 f(x) x x x1 CDF = Cumulative Distribution Function = P(x ≤ x1) x1 PDF = Probability Distribution Function = f(x ) P( x ≤ x1 ) = 0.6 also P( x ≤ ∞) = 1.0 P( x ≤ −∞) = 0 d (CDF ) dx CDF = f ( x ) dx f(x)= or ∫ So P( x ≤ x1 ) = x1 ∫ f ( x ) dx −∞ and P( x1 ≤ x ≤ x2 ) = x2 ∫ f ( x ) dx x1 JMS090800-14 AE3051 Normal/Gaussian Probability Distribution PDF(x) 1 σ 2π µ x (x − µ )2 1 f ( x) = exp − 2σ 2 σ 2π µ = mean σ2 = variance σ = standard deviation • Uses: – When N→∞ for a binomial distribution (e.g., for very large populations) – For events that are made up from many independent events each with any kind of distribution – For properties of samples when # samples is very large (e.g., sample means) – others… You You can can use use aa spreadsheet! spreadsheet! JMS090800-15 AE3051 Normal Distributions - Confidence Levels One Sigma: Two Sigma: Pr ob ( µ − σ ≤ x ≤ µ + σ ) = µ +σ +1 − −1 f (ξ ) dξ = ∫ f ( z )dz = 0.683 ∫ µ σ Pr ob ( µ − 2σ ≤ x ≤ µ + 2σ ) = µ + 2σ +2 −2 −2 f (ξ )dξ = ∫ f ( z )dz = 0.954 ∫ µ σ Three Sigma: Pr ob ( µ − 3σ ≤ x ≤ µ + 3σ ) = 0.997 Error Level Name Error Level Probable Error One Sigma 90% error “Two” Sigma Three Sigma Maximum Error Four Sigma Six Sigma ± 0.67 σ ±σ ± 1.65 σ ± 1.96 σ ±3σ ± 3.29 σ ±4σ ±6σ JMS090800-16 Prob. that Error is Smaller 50% 68% 90% 95% 99.7% 99.9 99.994% 99.9999999% Prob. that Error is Larger 1:2 ~ 1:3 1:10 1:20 1:370 1:1000 1:16000 1:1.01e9 Six Sigma is used for many electronic manufacturing processes AE3051 Sample Statistics • What if µ and σ are unknown (as is often the case)? – use estimates from measurements,x and s Mean Square N x Sample mean = x = ∑ i i =1 N N Sample variance = s = ∑ 2 i =1 Square of mean N 2 x i N ( xi − x )2 ∑ 2 i =1 = −x N −1 N N −1 – use (N-1) to compute average in s2 because we have N independent xi but if we also know xmean then we only need to know (N-1) xi to compute last remaining xi ! ⇒We call (N-1) the “degrees of freedom” for this calculation JMS090800-17 AE3051 Central Limit Theorem • One of the most important statistical theorems • Basis for most statistical methods commonly applied to measurements • Example: Suppose we measure a pressure 100 times and the avg. (mean) is 75 psi. Repeat test with 100 more p measurements and get 78 psi. Repeat many times (N). – Question: what is distribution for mean of average values? σ – Answer: Gaussian (normal) with σ x = N where σ is std dev of actual distribution of variable x. • This is also the distribution for any random variable that is the result of many independent random variables - no matter what the underlying distributions may be JMS090800-18 AE3051 Application: Confidence Intervals • Question: If one takes N readings and computes the sample average, how confident can you be that the average is really close to the true mean (µ)? • Confidence intervals are way to describe this; we know that the probability distribution of the sample mean for many different samples (N large) will be normal. Prob = c% that µ lies in shaded area defined by Example : c = 95%, x = µ ± ac σ x ac = 1.96, x = µ ± 1.96 σ x Answer 1 : 95% of x readings fall in µ ± 1.96 σ x µ − acσ x µ µ + acσ x Answer 2 : With 95% confidence, the true µ will fall within x ± 1.96 σ x JMS090800-19 AE3051 Application: Confidence Intervals - cont’d (1) • Since σ x = σ N where σ = true population standard deviation, we can write: Answer 3: 95% confidence limits : x ± 1.96 σ N • For large sample sizes (large N), we can approximate σ with sx so that we have: Answer 4 : 95% confidence limits : x ± 1.96 sx N EXAMPLE Find 95% confidence limits for x = 75 psi when Sx=8.3 psi for N = 50 samples. Answer : limits = x ± 1.96 S x 8.3 N = 75 ±1.96 = 75 ± 2.3 psi 50 Find 99.9% confidence limits for previous case (use previous table): Answer : limits = x ± 3.29 S x JMS090800-20 N = 75 ± 3.29 8.3 = 75 ± 3.9 psi 50 AE3051 Application: Confidence Intervals - cont’d (2) EXAMPLE (cont’d) How many N are required to assume that mean is in x ± 5% with 95% confidence? 8.3 = 75 ± (75 × 0.05) psi N or : N = 18.8 ≈ 19 samples Answer : limits = 75 ± 1.96 • Remarks – Above works only for N=large, or when σ is known – When N=small, then we cannot approximate σ by sx (due to N-1 in the denominator), and must treat σ as unknown • This leads to replacing the normal distribution with the “Student T distribution”, a subject beyond this introduction JMS090800-21 AE3051 Propagation of Uncertainty • • Many times experimental results are the result of several independent measurements combined using a theoretical formula. For mass flowrate p through a pipe: m& = ρuA = uπD 2 RT How do random and bias uncertainties in each variable contribute to whole? For the case where y=y(x1, x2, … xN) is a linear function, a statistical theorem states that: 2 2 ∂y ∂y ∂y σ xN σ x2 + L σ x1 + σ y = x x ∂ x ∂ ∂ 1 2 N 2 1 / 2 For uncertainties, ui, that are small compared to xi we can use a Taylor Series expansion in ui: y( x1 + u1 , x2 + u2 ,... x N + u N ) = y( x1 , x2 ,...x N ) + ∂y ∂y ∂y u1 + u2 + ... uN ∂x N ∂x1 ∂x2 Now, y is a linear function of the uncertainties and we can use the first equation to yield: ∂y ∂y ∂y u y = u1 + u2 + L u N ∂x1 ∂x2 ∂x N 2 JMS090800-22 2 2 1/ 2 AE3051 Examples of Uncertainty Propagation • Suppose the output is an additive combination: Then y = Ax1 + Bx2 ∂y ∂y = A and =B ∂x1 ∂x2 1/ 2 and 2 ∂y 2 ∂y u y = u x1 + ux2 ∂x1 ∂x2 1/ 2 2 2 = A2ux1 + B 2ux2 m • Suppose that the output is a multiplicative combination: m −1 Then and n ∂y mx1 x2 m x1 x2 y =A = A =m k k ∂x1 x1 x1 x3 x3 n m n ∂y y ∂y y =n and = −k x2 x3 ∂x2 ∂x2 2 2 u y u x1 u x2 ux3 = m + n + −k y x1 x2 x3 JMS090800-23 x x y = A 1 k2 x3 1/ 2 AE3051 Bias and Precision Uncertainties • • • • • We noted earlier that errors will include both bias (systematic) and precision (random) components These can usually be treated as independent and therefore the uncertainties for each can be combined into a total: [ ] 2 1/ 2 u y −total = u y − precision + u y −bias Note: if they are not independent, other combining in other ways may be necessary. SINGLE SAMPLE: if we are considering a single measurement and calculation of the dependent result (y), we must assume N=1 so that: uy σ y = y ± ac = y ± ac = y ± ac u y N 1 This affects only the precision error which is purely random so: 2 u y − single sample = ac u y = 1.96 u y [ u y −total − single sample = (ac u y − precision ) + u y −bias • 2 ] 2 1/ 2 Note: above based on 95% confidence interval (1.96 σ), ac=1.96 JMS090800-24 AE3051 Example #1: Uncertainty Calculation • L Consider determining the mass flowrate through a round pipe p Lπ p π v D = D m& = ρvA = RT ∆t 4 RT 4 2 • D 2 The fractional uncertainty in m & can then be developed using the previous formula for multiplicative combinations to be: u u u u u u = + + + + 2 m& p T L ∆t D 2 m& JMS090800-25 p 2 T 2 L 2 t 2 1/ 2 D AE3051 Example #1: Uncertainty Calculation - cont’d Assume the following uncertainties (ux-precision= sx) Variable Bias (ux/x) Precision (ux/x) p 0.55% 0.1% T 0.55% 0.4% ∆t 0.01% 2% L 0.1% D 1% Summed (Σu2)1/2 1.3% 2.0% Single-Sample 1.3% 4.0% Notes High precision pressure transducer, with 7.5-bit digitizer Medium precision temp. transducer, with 7.5-bit digitizer Accurate clock, but starting/stopping uncertainty of 0.01 sec in 0.5 sec measurement Only measured once with ruler having maximum 0.5 mm reading error over 0.5 m pipe length Only measured once with with ruler having maximum 0.5 mm reading error over 50 mm diameter ∆t meas. dominates precision error D meas. dominates bias error Precision error dominates Total uncertainty in single measurement of m& is then: um 2 2 1/ 2 = u precision + ubias = 4.2% m& & JMS090800-26 AE3051 Example #2: Uncertainty Calculation • Strain gage Consider a windtunnel drag transducer made by attaching a strain gage at the root of a cantilever beam as shown. A tip force, P, will produce a bending strain, εx, as shown. b h L εx = • • P Assume the electrical output of the strain gage circuit is e=Kεxe0 where e0 is the excitation voltage and K is the calib. factor to be determined. σx M y 6 P L = b = E EI b h2 E e e b h2 E K= = ε e0 e0 6 P L The uncertainty in K is then: u K ue ue0 ub uh u E u P u L = + + + 2 + + + K e e0 b h E P L 2 JMS090800-27 2 2 2 2 2 2 1/ 2 AE3051 Example #2: Uncertainty Calculation - cont’d Assume the following uncertainties: Variable e e0 b h E P L Uncertainty Single sample Bias (ux/x) 1% 0.5% 0.5% 0.5% 2% 1% Precision (ux/x) 0.2% 0 0 0 0 2% 0.2 2.65% 2.65% 0 2.01% 3.94% Notes Digital DVM yields good precision Only single initial measurement made same same same Bias reflects calibration errors while Precision is random error in readings Only single initial measurement made Total uncertainty in the measurement of K is then: [ ] uK 2 2 1/ 2 = u precision + ubias = 4.75% K JMS090800-28 AE3051 Wrap-up on Uncertainty Analysis • • Uncertainty analysis should always be a part of the design of any experiment – Avoid measurement processes that lead to greater uncertainty – Estimate the uncertainties in all measured and computed results Consider numerical precision in computational results (since even the best calculator doesn’t have infinite precision…) – Example: avoid taking differences between very large numbers – Example: s = ∑ ( x − x ) = 1 ∑ x − x N 2 N 2 i i =1 • N −1 N N i =1 2 i 2 N −1 Difference Difference between between large large numbers numbers Better Better BE CONSISTENT: – If the uncertainty in a computed result is 0.1%, then DO NOT include digits beyond 0.1% of the value! – Example: if computed result is 13,451.8793 with uncertainty of 0.1%, the number provided in the report should be: 13340. JMS090800-29 AE3051