Download Action Level

DQO Training Course
Day 1
Module 3
Key Concepts Underlying
DQOs and VSP
Presenter: Sebastian Tindall
120 minutes
(75 minute lunch break)
1 of 49
Key Points
Have fun while learning key statistical
concepts using hands-on illustrations
 This module prepares the way for a more indepth look at the DQO Process and the use of
VSP

2 of 49
The
Big
Picture
Schedule
Health
Risk
Sampling
Cost
Remediation
Cost
Decision
Error
Compliance
Waste
Disposal
Cost
3 of 49
Managing Uncertainty is a
Balancing Act
Unnecessary
Disposal
and/or
Cleanup
Cost
$
Sampling
and
Analyses
Cost
Threat
to Public
Health
and
Environment
Sampling
and
Analyses
Cost
$
$
$
PRP 1 Focus
Regulatory 1 Focus
4 of 49
Balance in Sampling Design
The statistician’s aim in designing surveys and
experiments is to meet a desired degree of
reliability at the lowest possible cost under the
existing budgetary, administrative, and physical
limitations within which the work must be
conducted. In other words, the aim is efficiency-the most information (smallest error) for the
money.
Some Theory of Sampling,
Deming, W.E., 1950
5 of 49
Our Methodology:
Use Hands-On Illustrations of...


Basic statistical concepts needed for VSP
and the DQO Process
Using...
Visual
Sample
Plan
6 of 49
Our Methodology:
Use Hands-On Illustrations of...


Basic statistical concepts needed for VSP
and the DQO Process
Using Coin flips
– Pennies
 Demo #1
 Demo #2
– Quarter
7 of 49
How Many Samples
Should We Take?
5?
50?
8 of 49
How Many Times Should I Flip a
Coin Before I Decide it is
Contaminated (Biased Tails)?
One tail, 50%
Two tails, 25%
Three tails, 12.5%
Four tails, 6%
Five tails, 3%
Six tails, 1.6%
Seven tails, 0.8%
Eight tails, 0.4%
Nine tails, 0.2%
Ten tails, 0.1%
9 of 49
Football Field
One-Acre
Football Field
30'0"
10 of 49
Example Problem
A 1-acre field was contaminated with mill
tailings in the 1960s
 Cleanup standard:
– “The true mean 226Ra concentration in the
upper 6” of soil must be less than 6.0 pCi/g.”
 There is a good chance that actual true mean
226Ra concentration is between 4.0 and 6.0
pCi/g

11 of 49
Example Problem (cont.)
Historical data suggest a standard deviation
of 1.6 pCi/g
 It costs $1000 to collect, process, and
analyze one sample
 The maximum sampling budget is $5,000

12 of 49
Simplified Decision Process
Take some number of samples
 Find the sample average 226Ra concentration
in our samples
 If we pass the appropriate QA/G-9 test, decide
the site is clean
 If we fail the appropriate QA/G-9 test, decide
the site is dirty

13 of 49
Marbles
Color
Ra-226, pCi/g
Clear
White
Green
Red
Dark Yellow
Blue
Black
3
4
5
6
7
8
9
14 of 49
Example of Ad Hoc Sampling
Design and the Results
Suppose we choose to take 5 samples for
various reasons: low cost, tradition,
convenience, etc.
 Need volunteer to do the sampling
 Need volunteer to record results
 We will follow QA/G-9 One-Sample t-Test
directions using an Excel spreadsheet

15 of 49
One-Sample t-Test Equation
from EPA’s Practical Methods
for Data Analysis, QA/G-9
Calculated t = (sample mean - AL)
-----------------------std. dev/sqrt(n)
If calculated t is less than table value,
decide site is clean
16 of 49
17 of 49
Comparing UCL to Action Level is Like Student’s t-Test
UCL = 4
X
4 - 6 = -2
UCL = 5
X
5 - 6 = -1
UCL = 7
X
7-6=1
UCL = 8
X
2
3
4
5
6
7
8-6=2
8
Action Level
True Mean 226Ra Concentration
18 of 49
Learn the Jargon
• t-test
• UCL - upper confidence
limit
• AL - action level
• N - target population
• n - population units
sampled
•  - population mean
• x - sample mean
•  - population standard
deviation
• s - sample standard
deviation
•
•
•
•
•
•
•
•
•
•
Frequency distribution
Histograms
H0 - null hypothesis
 - Alpha error rate
 - Beta error rate
Gray Region
LBGR
 - width of Gray Region
Coefficient of Variation
Relative Standard Deviation
19 of 49
t-test
Calculated t = (sample mean - AL)
-----------------------(s / n )
If calculated t is less than table value, decide
site is clean
20 of 49
Upper Confidence Limit, UCL
For a 95% UCL and assuming sufficient n:
If you repeatedly calculate sample means for many independent
random sampling events from a population, in the long run, you
would be correct 95% of the time in claiming that the true mean
is less than or equal to the 95% UCL of all those sampling events.
Note: Different X s will produce different UCLs
UCL  X  [ t1 , df * (s/ n )]
UCL X [Z
*(s/ n )]
1
21 of 49
Upper Confidence Limit, UCL
More commonly, but some experts dislike:
For a single, one-sided UCL, you are 95% confident
that the true mean is less than or equal to your
calculated UCL.
(The true mean is bracketed by, in our case, is
usually zero) and the UCL.)
(See Hahn and Meeker in Statistical Intervals A
Guide for Practitioners, p. 31).
22 of 49
Action Level
A measurement threshold value of the Population
Parameter (e.g., true mean) that provides the criterion for
choosing among alternative actions.
23 of 49
N
Target Population: The set of N population units about which
inferences will be made
Population Units: The N objects (environmental units) that
make up the target or sampled population
n
The number of population units selected and measured is n
24 of 49
10 x 10 Field
Population = All 100 Population Units
25 of 49
10 x 10 Field
Population = All 100 Population Units
Sample = 5 Population Units
1.5
1.9
2.3
1.7
1.5
26 of 49
Population Mean

The average of all N population units
 
1
N

N
i=1
Xi
Sample Mean
X
The average of the n population units actually measured
1
X 
n
 Xi
n
i=1
27 of 49
Population Standard Deviation

The average deviation of all N population units from the
population mean
N

 Xi 
i 1
 2
N
Sample Standard Deviation
s
The “average” deviation of the n measured units from the
sample mean

i 1
n
s
 Xi  X
2
n 1
28 of 49
Spatial Distribution - Football Field
29 of 49
Probability Density Function
30 of 49
SHOW Histogram File
31 of 49
SHOW VDT Step by Step
Histogram File
32 of 49
The Null Hypothesis
H0
The initial assumption about how the
true mean relates to the action level
Example: The site is dirty. (We’ll
assume this for the rest of this
discussion)
H 0 :   Action Level
33 of 49
The Alternate Hypothesis
HA
The alternative hypothesis is
accepted only when there is
overwhelming proof that the Null
condition is false.
H A :   Action Level
34 of 49
Null Hypothesis = Site is Dirty
The Alpha Error Rate
(on Type 1 or False + errors)

The chance of deciding that a dirty site is clean
when the true mean is greater than or equal to
the action level
35 of 49
The Alpha Error Rate
(on Type 1 or False + Errors)
(Null Hypothesis = Site is Dirty)
α
A false positive decision or Type 1 error occurs when a decisionmaker rejects the null hypothesis (calls it false) when H0 is
actually true. The size of the error is expressed as a probability,
usually referred to as Alpha (. This error occurs when the data
(sample result x-bar or UCL) indicates that the site is clean
when the true mean is actually at or above the Action Level.
In other words, the Alpha error is the probability that your
sample result is below the Action Level when the true means is
actually at or above the Action Level. That probability is
usually set to between 1-5%.
36 of 49
The Alpha Error Rate
(on Type 1 or False + Errors)
(Null Hypothesis = Site is Clean)
α
A false positive decision or Type 1 error occurs when a decisionmaker rejects the null hypothesis (calls it false) when H0 is
actually true. The size of the error is expressed as a probability,
usually referred to as Alpha (. This error occurs when the data
(sample result x-bar or UCL) indicates that the site is dirty when
the true mean is actually at or below the Action Level. In
other words, the Alpha error is the probability that your sample
result is above the Action Level when the true mean is at or
below the Action Level. That probability is usually set to
between 5-1%.
37 of 49
Null Hypothesis = Site is Dirty
The Beta Error Rate
(on Type 2 or False - errors)

The chance of deciding a clean site is dirty
when the true mean is equal to the lower
bound of the gray region (LBGR)
38 of 49
The Beta Error Rate
(on Type 2 or False – Errors)
(Null Hypothesis = Site is Dirty)
β
A false negative decision or Type 2 error occurs when a decisionmaker accepts the null hypothesis (calls it true) when H0 is
actually false. The size of the error is expressed as a probability,
usually referred to as Beta (β. This error occurs when the data
(sample result x-bar or UCL) indicates that the site is dirty when
the true mean is actually below the Action Level. In other
words, the Beta error is the probability that your sample result is
at or above the Action Level when the true mean is actually
below the Action Level. That probability is negotiated and set to
between 1-50%.
39 of 49
The Beta Error Rate
(on Type 2 or False – Errors)
(Null Hypothesis = Site is Clean)
β
A false negative decision or Type 2 error occurs when a decisionmaker accepts the null hypothesis (calls it true) when H0 is
actually false. The size of the error is expressed as a probability,
usually referred to as Beta (β. This error occurs when the data
(sample result x-bar or UCL) indicates that the site is clean
when the true mean is actually above the Action Level. In
other words, the Beta error is the probability that your sample
result is at or below the Action Level when the true mean is
actually above the Action Level. That probability is negotiated
and set to between 1-20%.
40 of 49
Evaluate Alpha & Beta Errors
µ:α
µ:β
Action
Level
LBGR
Alpha
Error
Beta
Error
0
75
100
∞
True Mean Concentration
41 of 49
Gray Region
Gray Region = AL – LBGR
A range of values of the population parameter of
interest (such as the true mean contaminant
concentration, ) where the consequences of
making a decision error are relatively minor.
42 of 49
Gray Region & LBGR
Gray Region = AL – LBGR
The Gray Region is bounded on one side by the
action level, and on the other side by the parameter
value where the consequences of decision error begins
to be significant. This point is labeled LBGR, which
stands for Lower Bound of the Gray Region.
43 of 49
The Width of Gray Region
(Null Hypothesis = Site is Dirty)
= AL – 1
Width of GR = AL – LBGR
The Lower Bound of the Gray Region (1)
is defined as the hypothetical true mean
concentration where the site should be declared
clean with a reasonably high probability.
44 of 49
The Width of Gray Region
(Null Hypothesis = Site is Clean)
= 1 – AL
Width of GR = UBGR – AL
The Upper Bound of the Gray Region (1)
is defined as the hypothetical true mean
concentration where the site should be declared
dirty with a reasonably high probability.
45 of 49
Coefficient of Variation:
CV = s / x-bar
If CV > 1, not Normal
Relative Standard Deviation:
RSD (%) = CV * 100
If RSD > 100%, not Normal
46 of 49
SHOW VST File
for
Coefficient of Variation
and
RSD
47 of 49
Summary
Decisions about population parameters, such
as the true mean, , and the true standard
deviation, , are based on statistics such as
the sample mean, X , and the sample standard
deviation, s. Since these decisions are based
on incomplete information, they will be in
error.
48 of 49
End of Module 3
Thank you
Questions?
We will now take a 75 minute lunch break.
Please be back in 1 hour and 15 minutes.
49 of 49