Download Repeatability Pooling Variances

Repeatability –
Pooling Variances
Stanley N. Deming, Ph.D.
Statistical Designs
9941 Rowlett Road, Suite 10
Houston, TX 77075-3404 U.S.A.
713 947 1551
[email protected]
BEBPA Nice 28 September 2011. DO NOT COPY Copyright © 2011 by Stanley N. Deming. All rights reserved. Slide 1
Pooling: Fundamental Definition Of Variance
An estimated variance, s2, is a sum of
squares (SS) divided by a degrees of
freedom (df):
sum of squares
s  variance 
degrees of freedom
For example, the familiar correct equation
for the estimated standard deviation of n
data points about their mean is
n
2
An estimated standard deviation, s, is the
square root of an estimated variance, s2:
s
 ( xi  x )2
i 1
n 1
where the numerator is
n
s
s2 
variance 
SS
df
 ( xi  x )2  sum of squared deviations
i 1
and the denominator is
n  1  degrees of freedom of deviations
Because one degree of freedom has been
used to calculate the mean, only n - 1 of
the deviations are independent.
BEBPA Nice 28 September 2011. DO NOT COPY Copyright © 2011 by Stanley N. Deming. All rights reserved. Slide 2
Pooling: A Measurement Method As A System
A measurement method may be represented
with a systems theory diagram.
The "Measurement
Measurement Method"
Method converts
information contained in the sample into
numerical results.
The "Sample"
Sample (the input) is usually a physical
material to be characterized.
The "Results" (the output) are a set of data.
For our purposes here, the "measurement
method" may be defined as a bioassay.
A caution about words: Statisticians call the
results a "statistical sample," a subset of a
few of the conceptually infinite number of
individual results that could have been
obtained from the "physical sample."
The physical sample on the left in the figure
is not the same as the statistical sample on
the right in the figure. (Don’t use the word
"
"sample"
l " iin a conversation
i with
iha
statistician without putting an adjective in
front of it!)
BEBPA Nice 28 September 2011. DO NOT COPY Copyright © 2011 by Stanley N. Deming. All rights reserved. Slide 3
Pooling: Common Measurement Practice
Many measurement laboratories make
multiple measurements on each sample that
is submitted. These are called replicate
measurements — they are repeated
measures of the same thing.
The figure at the right shows three replicate
measurement results (the three circles at the
right, n = 3) for a single sample of material.
Many measurement courses (undergraduate
qua t tat e a
quantitative
analysis,
a ys s, for
o e
example)
a p e) po
pointt out
that it requires at least two replicate
measurements to calculate a standard
deviation, and taking three measurements on
each p
physical
y
sample
p is a common p
practice.
BEBPA Nice 28 September 2011. DO NOT COPY Copyright © 2011 by Stanley N. Deming. All rights reserved. Slide 4
Pooling: The Mean And Standard Deviation For One Sample
Suppose the three replicate measurements
give the values 61.5, 63.3, and 62.7.
These three values are graphed at the bottom
right. The vertical dimension has no meaning
here — it is just a convenient place to put the
data points.
The average is
x 
61.5  63.3  62.7
 62.50
3
and the standard deviation s is
3
 ( xi  62.50)2
i 1
s
3 1
 0.9165
This estimate of  has only 2 degrees of
freedom.
BEBPA Nice 28 September 2011. DO NOT COPY Copyright © 2011 by Stanley N. Deming. All rights reserved. Slide 5
Pooling: The Mean And Standard Deviation For One Sample
In the graph at the bottom right, some lines
have been added.
The vertical line is drawn at 62.50, the mean
of the three data points.
For each data point, a horizontal line has
been drawn from that data point to the mean.
These horizontal lines represent the residuals
that are squared and summed to calculate the
standard
sta
da d de
deviation.
at o
The results are reported to the client:
mean
std. dev.
n
=
=
=
62.50
0.9165
3
((Reporting
p
g 62.50  0.92 is discouraged.)
g )
So much for the first sample.
BEBPA Nice 28 September 2011. DO NOT COPY Copyright © 2011 by Stanley N. Deming. All rights reserved. Slide 6
Pooling: A Second Sample
The figure at the right shows three replicate
measurement results for a second physical
sample of material (three more circles at the
right).
BEBPA Nice 28 September 2011. DO NOT COPY Copyright © 2011 by Stanley N. Deming. All rights reserved. Slide 7
Pooling: A Second Sample
Suppose this second sample gives measured
values of 66.9, 67.2, and 66.3.
These three values are graphed at the bottom
right, along with the data from the first
sample. Again, the vertical dimension has no
meaning.
The average is
x 
66.9  67.2  66.3
 66.80
3
and the standard deviation s is
3
s
 ( xi  66.80)2
i 1
3 1
 0.4583
This estimate of  has only 2 degrees of
freedom.
BEBPA Nice 28 September 2011. DO NOT COPY Copyright © 2011 by Stanley N. Deming. All rights reserved. Slide 8
Pooling: A Second Sample
In the graph at the bottom right, some lines
have been added.
The vertical line is drawn at 66.80, the mean
of the three data points.
For each data point, a horizontal line has
been drawn from that data point to the mean.
These horizontal lines represent the residuals
that are squared and summed to calculate the
standard
sta
da d de
deviation.
at o
The results are reported to the client:
mean
std. dev.
n
=
=
=
66.80
0.4583
3
So much for the second sample.
p
BEBPA Nice 28 September 2011. DO NOT COPY Copyright © 2011 by Stanley N. Deming. All rights reserved. Slide 9
Pooling: The Source Of Variation For The Two Samples
The two means (62.50 and 66.80) differ from
each other because of the different
characteristics of the two physical samples.
Both sets of results came from the same
measurement method. If the samples are
homogeneous, any variation of the individual
results around their separate means must be
caused by the measurement method.
The standard deviation is a characteristic
of the
o
t e measurement
easu e e t method,
et od, not
ot the
t e
sample!
Using a new subscript to indicate the physical
sample
p number,, we would therefore expect
p
the
( x1i  x1)
deviations to be similar to the
( x2i  x2 )
deviations.
BEBPA Nice 28 September 2011. DO NOT COPY Copyright © 2011 by Stanley N. Deming. All rights reserved. Slide 10
Pooling: The Source Of Variation For The Two Samples
Further, the two sums of squares
n1
 ( x1i  x1)2
i 1
and
n2
 ( x2i  x2 )2
i 1
should be two separate
p
estimates of the
variability of the measurement method.
One estimate has n1 - 1 df, and the other
estimate has n2 - 1 df.
BEBPA Nice 28 September 2011. DO NOT COPY Copyright © 2011 by Stanley N. Deming. All rights reserved. Slide 11
Pooling: The Pooled Standard Deviation
Using all of the information from the two
samples, the composite (or pooled) sum of
squared deviations from the means is
n1
n2
i 1
i 1
 ( x1i  x1)2   ( x2i  x2 )2
and the composite
p
((or p
pooled) degrees
g
of
freedom is
(n1 - 1) + (n2 - 1) = n1 + n2 – 2
Returning
R
t i tto th
the ffundamental
d
t ld
definition
fi iti off
variance, and substituting the pooled sum of
squares and pooled degrees of freedom gives
n1
sp 
n2
 ( x1i  x1)2   ( x2i  x2 )2
i 1
i 1
n1  n2  2
Because it has more degrees of freedom, sp
is a better estimate of  .
BEBPA Nice 28 September 2011. DO NOT COPY Copyright © 2011 by Stanley N. Deming. All rights reserved. Slide 12
Pooling: Generalizing To Many Statistical Samples
If j is an index that refers to a particular
statistical sample, then the notation
2

j=1
indicates a summation over two statistical
samples. The equation for the pooled
standard deviation
n1
sp 
n2
 ( x1i  x1)2   ( x2i  x2 )2
i 1
i 1
n1  n2  2
can be rewritten
2 nj
sp 
 ( x ji  x j )2
j 1 i 1
2


 nj   2
 j 1 
BEBPA Nice 28 September 2011. DO NOT COPY Copyright © 2011 by Stanley N. Deming. All rights reserved. Slide 13
Pooling: The Method Standard Deviation, m
In general, if J represents the total number of
statistical samples, then
J
 m  sp 
nj
 ( x ji  x j )2
j 1 i 1
J


 nj   J
 j 1 
Many laboratories make multiple
measurements on each physical sample that
is analyzed (see the figure at the right). This
historical information can be used to calculate
a pooled standard deviation. Because the
number of degrees of freedom is so large
(e.g., 19,054), sp becomes a very good
estimate of  for the measurement method.
This population parameter is given the
subscript "m"
m and is called a "method
method
standard deviation": m = 0.8159, for
example.
BEBPA Nice 28 September 2011. DO NOT COPY Copyright © 2011 by Stanley N. Deming. All rights reserved. Slide 14
Pooling: An Alternative Protocol For Measurement
When m is known, it isn't necessary to make
multiple measurements. Future samples can
be measured only once.
In the past, measurements were reported
using the mean and the standard deviation
calculated for each sample:
x1  62.50
s1  0.9165
n3
x2  66.80
s2  0.4583
n3
In the future, measurements can be reported
using a mean of only one measurement and
the method standard deviation
x9,528  63.74
x9,529  58.67
 m  0.8159
 m  0.8159
n 1
n 1
This saves the cost of extra measurements
and gives the customer a consistent (and
more realistic) estimate of the uncertainty.
(Scared to do this? Think again. When you
have your cholesterol measured, how many
measurements does your health care plan
pay for? They know their m.)
BEBPA Nice 28 September 2011. DO NOT COPY Copyright © 2011 by Stanley N. Deming. All rights reserved. Slide 15
Pooling: An Alternative Protocol For Measurement
The confidence interval of the mean does
decrease with multiple measurements:
x
z m
n
When n = 3, the confidence interval is about
42% narrower than when n = 1: tripling the
cost is a big price to pay for a slight narrowing
in the confidence interval.
If the data are proportionally heteroscedastic,
a a
an
analogous
a ogous app
approximate
o
ate equat
equation
o ca
can be
used to pool the relative standard deviation:
2
 x ji  x j 

 x

j
m

j 1 i 1 


 J

 nj   J
 j 1 
However, if the measured values are small
and have a homoscedastic component, this
equation will give an inflated estimate.
J
nj
BEBPA Nice 28 September 2011. DO NOT COPY Copyright © 2011 by Stanley N. Deming. All rights reserved. Slide 16
Pooling: An Alternative Calculation
When the standard deviations of the individual
samples have already been calculated, an
alternative equation for calculating the pooled
standard deviation can be used:
sp 
(n1  1)s12  (n2  1)s22
n1  n2  2
Recall that an estimated variance is a sum of
squares divided by its degrees of freedom.
Multiplying an estimated variance by its
degrees of freedom will
ill "regenerate" the ssum
m
of squares. For example,
n1
s12 
 ( x1i  x1)2
i 1
(n1  1)
; (n1  1)s12 
n1
 ( x1i  x1)2
i 1
If the reported values of the standard
deviation have been rounded (e
(e.g.,
g 0
0.5
5
instead of 0.4582575...), this alternative
calculation might give distorted results.
BEBPA Nice 28 September 2011. DO NOT COPY Copyright © 2011 by Stanley N. Deming. All rights reserved. Slide 17
Pooling: Where Does The Variance Come From?
The "measurement method" might be larger
than has been assumed up to this point. In
bioassays, it might be more than just the plate
on which reagents have been added and
dilutions have been made.
Just what is meant by "the measurement
method depends on the definition of the
method"
boundaries of the "measurement system."
Thus hairs are split about "repeatability,"
"intermediate
te ed ate p
precision,"
ec s o , etc.
etc
Nonetheless, once the "measurement
method" and its sources of variation has been
defined,, the resulting
g replicate
p
data can be
pooled as shown in this presentation.
(Nested, split-plot, or staggered-nested
experimental
p
designs
g can be used to estimate
individual sources of variation. But that’s
another story.)
BEBPA Nice 28 September 2011. DO NOT COPY Copyright © 2011 by Stanley N. Deming. All rights reserved. Slide 18
Repeatability –
Pooling Variances
(END)
Stanley N. Deming, Ph.D.
Statistical Designs
9941 Rowlett Road, Suite 10
Houston, TX 77075-3404 U.S.A.
713 947 1551
[email protected]
BEBPA Nice 28 September 2011. DO NOT COPY Copyright © 2011 by Stanley N. Deming. All rights reserved. Slide 19