Download Quantitative Testing Plans

Quantitative Testing Plans
May 8-10, 2006
Iowa State University, Ames – USA
Jean-Louis Laffont
Kirk Remund
Overview
• Statistical framework
• Implementation in Seedcalc
ISTA Statistics Committee
2
Testing plan design: statistical framework for quantitative methods
• Suppose n pools of m seeds were taken from the lot
and that J flour sub-samples from each pool were
measured K times.
Seed lot:
true AP% = p
Grinding
seeds into
flour
…
…
J flour
sub-samples
per pool
Measurement
n pools of
m seeds
…
…
yijk
K measures
per flour
sub-sample
Measure 1 Measure 2
Pool i
…
Measure K
Flour sub-sam ple 1
Flour sub-sam ple 2
…
Flour sub-sam ple J
ISTA Statistics Committee
3
Testing plan design: statistical framework for quantitative methods
• Model:
y ijk  p  a i  b j(i )  e ijk
Measurement
k
made on flour
sub-sample j
from pool i
=
True
AP%
+
Random
effect of
pool i

2
N 0,  sampling

+
Random effect
of flour
sub-sample
j from pool i

2
N 0,  flour

+
Random effect
of measurmnt k
for flour
sub-sample j
from pool i

N 0,  2measuremen t
The parameter p is estimated by the sample mean:
1
2

p̂ 
y
~
N
p,


ijk
p̂ 
nJK i , j,k
 2 sampling  2 flour  2 measuremen t
 


n
nJ
nJK
2
p̂
ISTA Statistics Committee
4

Testing plan design: statistical framework for quantitative methods
Overall distribution of y ~ N(p,  2 y )
yijk
 2 sampling  2 flour  2 measuremen t
 


n
nJ
nJK
2
p̂
0.0
0.5
1.0
1.5
2.0
Sampling n pools of m seeds
AP%
2
: derived from the
sampling
variance of B(m, p ker nel )
p +

2
sampling
p ker nel (1  p ker nel )

m
Flour sub-sampling
-1.0
-0.5
0.0
0.5
1.0
2
 flour
AP%
 2 flour and  2 measuremen t
are obtained from
historical experiments
Measurement
 2measuremen t
ISTA Statistics Committee
5
Testing plan design: statistical framework for quantitative methods
• Remember that the true AP% p in the lot is expressed
in %DNA when using quantitative methods
p ker nel (1  p ker nel )
2
and that  sampling 
is expressed on a
m
kernel basis.
 Introduction of the b-Factor (biological factor) to
convert from %DNA to %Seed units:
%Seed = b-Factor x %DNA
or
pker nel  b  p
• Examples:
• Reference material and test lots have the same zygosity/ploidy/copy
number  b-Factor= 1
• Homozygous reference material and hemizygous test lots
 b-Factor= 2
ISTA Statistics Committee
6
Testing plan design: statistical framework for quantitative methods
• Re-expression of

2

2
• Re-expression of 
2
sampling
sampling
p̂
in %DNA units:
p(1  bp)

bm
2 sampling 2 flour  2 measuremen t



n
nJ
nJK
:
Having observed in some experiments that ²measurement
seems to depend on p, the true AP probability, while
CVmeasurement is fairly constant, we can rewrite 2 p̂ as:
 2 p̂
p(1  bp)  2flour (pCVmeasurement )2



bnm
nJ
nJK
ISTA Statistics Committee
7
Testing plan design: statistical framework for quantitative methods
• Lets now define an Acceptance Limit (AL) such that:
• if p̂  AL, “accept” the lot
• if
p̂
> AL, “reject” the lot
• We can then calculate the probability to “accept” the lot,
given a true unknown AP% p:
 p̂  p AL  p 
 AL  p 
where F is the cumulative distribution

Pr( p̂  AL | p)  Pr

| p   F
 

   function for the standard normal distribution
 p̂
p̂
 p̂



• This formula serves as a basis for elaborating an
OC curve that can be used to investigate properties
of a testing plan
ISTA Statistics Committee
8
Testing plan design: statistical framework for quantitative methods
• Example: testing plan components: 2 pools of 3000 seeds,
100
1 flour sub-sample/pool, 3 measurements/flour sub-sample,
Std-Dev of flour sub-sampling error = 0.011%, measurement CV = 15%,
Acceptance Limit (AL) = 0.1%
(lot « accepted » if average of the 2 x 1 x 3 readings is  AL)
60
40
20
5%
0
Probability of acceptance (%)
80
95%
0.0
0.1
0.2
0.3
0.4
0.5
True AP% in lot
ISTA Statistics Committee
9
Testing plan design: statistical framework for quantitative methods
• Consumer risk and producer risk are given respectively
by:
 AL  LQL 

Consumer risk  Pr( p̂  AL | LQL)  F



p̂


 AL  AQL 

Pr oducer risk  Pr( p̂  AL | AQL)  1  F



p̂


where F is the cumulative distribution
function for the standard normal distribution
ISTA Statistics Committee
10
Testing plan design: implementation for quantitative methods
• All of the methods discussed have been implemented in
the newest version of the Microsoft Excel® spreadsheet
Seedcalc
Estimating AP%
Designing
testing plans
ISTA Statistics Committee
Comparing
testing plans
11
Testing plan design: implementation for quantitative methods
Testing plan design
LQL, AQL
and AL
Enter n, m,
J, K and …
historical
assay
variation
and…
b-Factor
and …
and get consumer
and producer risks
and OC curve
ISTA Statistics Committee
12
Testing plan design: implementation for quantitative methods
The « Find Plan » tool can
help the user to find
testing plans satisfying
certain conditions given
some parameters
ISTA Statistics Committee
13
Testing plan design: implementation for quantitative methods
Parameters for the
search algorithms
ISTA Statistics Committee
14
Testing plan design: implementation for quantitative methods
Find the highest AL that
meets target consumer
risk for the LQL.
No consideration of the
producer risk target.
n, m, J and K are held
fixed
ISTA Statistics Committee
15
Testing plan design: implementation for quantitative methods
Consumer and producer
risk targets satisfied
by changing AL, n, I and J
ISTA Statistics Committee
16
Testing plan design: implementation for quantitative methods
Consumer and producer
risk targets satisfied
by changing AL, m, I and J
ISTA Statistics Committee
17
Testing plan design: implementation for quantitative methods
Consumer and producer
risk targets satisfied
by changing AL, n, m, I and J
ISTA Statistics Committee
18
Testing plan design: implementation for quantitative methods
Compare plans
Visual comparison of OC curves
along with testing plan parameters
ISTA Statistics Committee
19
Example
Historical data gave 0.15% for the estimate of the flour standard-deviation.
We expect that the measurement CV range is from 10% to 30% and we
consider the following testing plan:
. LQL = 0.7% for a consumer confidence = 95%
. AQL = 0.15% for a producer confidence = 95%
. 1 pool of 3000 seeds, 2 flour sub-samples, 3 measurements
. AL = 0.39%
1. Does this plan meet consumer and producer requirements when the
measurement CV = 10%?
2. Compare the outcomes of this testing plan when the CV is varying from
10% to 30%.
ISTA Statistics Committee
20
Example
1. Does this plan meet consumer and producer requirements when the
measurement CV = 10%?
YES
ISTA Statistics Committee
21
Example
2. Compare the outcomes of this testing plan when the CV is varying from
10% to 30%.
ISTA Statistics Committee
22