Download Ch2_Simple_Comparative_Experiments

Chapter 2: Simple Comparative
Experiments (SCE)
• Simple comparative experiments: experiments
that compare two conditions (treatments)
– The hypothesis testing framework
– The two-sample t-test
– Checking assumptions, validity
1
Portland Cement Formulation (page 23)
• Average tension bond sterngths (ABS)
differ by what seems nontrivial amount.
• Not obvois that this difference is large
enough imply that the two formulations
really are diff.
• Diff may be due to sampling fluctuation and
the two formulations are really identical.
• Possibly another two samples would give
opposite results with strength of MM
exceeding that of UM.
• Hypothesis testing can be used to assist in
comparing these formulations.
• Hypothesis testing allows the comparison to
be made on objective terms, with knowledge
of risks associated with searching the wrong
conclusion
2
Graphical View of the Data
Dot Diagram, Fig. 2-1, pp. 24
• Response variable is a random variable
• Random variable:
1. Discrete
2. continuous
3
Box Plots, Fig. 2-3, pp. 26
• Displays min,
max, lower and
upper quartile,
and the median
• Histogram
4
Probability Distributions
• Probability structure of a
Random variable, y, is
described by its probability
distribution.
• y is discrete: p(y) is the
probability function of y (F24a)
• y is continuous: f(y) is the
probability density function
of y (F2-4b)
5
Probability Distributions
Properties of probability distributions
• y-discrete:
0  p(y j )  1
all values of y j
P ( y  y j )  p ( y j ) all values of y j

p(y j )  1
all values of y j
f (y )  0
• y-continuous:
b
P (a  y  b )   f ( y )dy
a

 f ( y )dy  1

6
Probability Distributions
mean, variance, and expected values

  yf ( y )dy y continuous
  E ( y )  
  yp ( y ) y discrete
 all y

2
(
y


)
f ( y )dy y continuous



V ( y )   2  E ( y   ) 2   
  ( y   ) 2 p ( y ) y discrete
 all y
7
Probability Distributions
Basic Properties
1.
2.
3.
4.
5.
6.
E(c) = c
E(y) = 
E(cy) = c E(y) = c 
V(c) = 0
V(y) = 2
V(cy) = c2 V(y) = c2 2
8
Probability Distributions
Basic Properties
• E(y1+y2) = E(y1)+E(y2) = 1+2
V ( y 1  y 2 ) V ( y 1 ) V ( y 2 )  2Cov ( y 1 , y 2 )
• Cov(y1,y2) = E[(y1-1)(y2-2)]
• Covariance: measure of the linear
association between y1 and y2.
• E(y1.y2) = E(y1).E(y2) = 1.2 (y1 and y2 are
indep)
y1
E (y 1)
E( )
y2
E (y 2 )
9
Sampling and Sampling
Distributions
• The objective of statistical inference is to draw conclusions
about a population using a sample from that population.
• Random Sampling: each of N!/(N-n)!n! samples has an
equal probability of being chosen.
• Statistic: any function of observations in a sample that does
not contain unknown parameters.
• Sample mean and sample variance are both statistics.
n
y 
y
i 1
n
n
i
,
S2 
 y
i 1
i
y
2
n 1
10
Properties of sample mean and variance
• Sample mean is a point estimator of population mean 
• Sample variance is a point estimator of population variance
2
• The point estimator should be unbiased. Long run average
should be the parameter that is being estimated.
• An unbiased estimator should have min variance. Min
variance point estimator has a variance that is smaller than
the variance of any other estimator of that parameter.
1
E (S 2 ) 
E (SS ),
E(SS)  (n  1) 2
n 1
11
Degrees of freedom
• (n-1) in the previous eq is called the NDOF
of the sum of squares.
• NDOF of a sum of squares is equal to the
no. of indep elements in the sum of squares
n
• Because   y i  y   0 , only (n-1) of the n
i 1
elements are indep, implying that SS has (n1) DOF
12
The normal and other sampling
distributions
• Normal Distribution
1
 (1/ 2)[( y   ) /  ]2
f (y ) 
e
,
 2
  y  
• y is distributed normally with mean  and
variance 2 y N (  ,  )
• Standard normal distribution: =0 and 2=1
z
N (0,1)
z 
y 

13
Central Limit Theorem
• If y1, y2, …, yn is a sequence of n independent
and identically distributed random variables
with E(yi) =  and V(yi) = 2 (both finite) and x
x  n
= y1+y2+…+yn, then z n 
has an
2
n
approximate N(0,1) distribution.
• This implies that the distribution of the sample
averages follows a normal distribution with
• This approximation require a relatively large
sample size (n≥30)
14
Chi-Square or c2 distribution
• If z1, z2, …, zn are normally and
independently distributed random variables
with mean 0 and variance 1 NID(0,1), then
the random variable x  z 12  z 22  ...  z 32
follows the chi-square distribution with k
DOF.
1
f (x ) 
2
k /2
k 
 
2
x
( k / 2) 1  x / 2
e
,
x 0
15
Chi-Square or c2 distribution
• The distribution is asymmetric (skewed)
=k
2= 2k
• Appendix III
16
Chi-Square or c2 distribution
• y1, y2, …, yn is a random sample from
N(,), then
n
SS
2

 y
i 1
i
y
2
2
c n21
• SS/2 is distributed as chi-square with n-1
DOF
17
Chi-Square or c2 distribution
• If the observations in the sample are
NID(,), then the distribution of S2 is
 2 /(n  1)  c n21
and,
,,,
• Thus, the sampling distribution of the
sample variance is a constant times the chisquare distribution if the population is
normally distributed
18
Chi-Square or c2 distribution
• Example:
The Acme Battery Company has developed a new cell
phone battery. On average, the battery lasts 60 minutes on
a single charge. The standard deviation is 4.14 minutes.
a) Suppose the manufacturing department runs a quality
control test. They randomly select 7 batteries. The standard
deviation of the selected batteries is 6 minutes. What would
be the chi-square statistic represented by this test?
b) If another sample of 7 battery was selected, what is the
probability that the sample standard deviation is greater
than 6?
DOX 6E Montgomery
19
Chi-Square or c2 distribution
Solution
• a) We know the following:
– The standard deviation of the population is 4.14 minutes.
– The standard deviation of the sample is 6 minutes.
– The number of sample observations is 7.
• To compute the chi-square statistic, we plug these data in
the chi-square equation, as shown below.
 c2 = [ ( n - 1 ) * s 2 ] / σ 2
c2 = [ ( 7 - 1 ) * 62 ] / 4.142 = 12.6
DOX 6E Montgomery
20
Chi-Square or c2 distribution
• b) To find the probability of having a sample
standard deviation S > 6, we refer to the Chisquare distribution tables, we find the value of a
corresponding to chi-square = 12.6 and 6 degrees
of freedom.
• This will give a  0.05 which is the probability of
having S > 6
DOX 6E Montgomery
21
DOX 6E Montgomery
22
t distribution with k DOF
• If z and c k2 are indpendent normal and chisquare random variables, the random
variable
z
tk 
c k2 / k
• Follow t distribution with k DOF as
follows:
[(k  1) / 2]
1
f (t ) 
,  t  
2
( k 1) / 2
k  (k / 2) [(t / k )  1]
23
t distribution with k DOF
 = 0 and 2 = k/(k-2) for k>2
• If k=infinity, t becomes standard normal
• If y1, y2, …, yn is a random sample from
N(,), then
y 
tk 
S/ n
is distributed as t with n-1 DOF
24
t distribution - example
Example:
• Acme Corporation manufactures light bulbs. The
CEO claims that an average Acme light bulb lasts
300 days. A researcher randomly selects 15 bulbs
for testing. The sampled bulbs last an average of
290 days, with a standard deviation of 56 days. If
the CEO's claim were true, what is the probability
that 15 randomly selected bulbs would have an
average life of no more than 290 days?
DOX 6E Montgomery
25
t distribution – Example
Solution
To find P(x-bar<290),
• The first thing we need to do is compute the t
score, based on the following equation:
– t = [ x - μ ] / [ s / sqrt( n ) ]
t = ( 290 - 300 ) / [ 56 / sqrt( 15) ] = - 0.692
• P(t<-0.692) is equivalent to P(t>0.692)
DOX 6E Montgomery
26
t distribution – Example
• From the t-distribution tables, we have a =
25% which correspond to the probability of
having the sample average less than 290
DOX 6E Montgomery
27
DOX 6E Montgomery
28
F distribution
• If c and c are two independent chisquare random variables
with
u
and
v
DOF,
2
c
then the ratio Fu ,v  u2 / u
cv / v
• Follows the F dist with u numerator DOF
and v denominator DOF
2
u
2
v
(u / 2) 1
 u  v  u 
(u / 2) 1

x
 
2  v 

h (x ) 
, 0x 
(u v ) / 2
 u   v  u

       x  1
 2   2  v

29
F distribution
• Two independent normal populations with
common variance 2. If y11, y12, …, y1n1 is a
random sample of n1 observations from 1st
population and y21, y22, …, y2n2 is a random
sample of n2 observations from 2nd
population, then S 12  F
S
2
2
n 11, n 2 1
30
The Hypothesis Testing Framework
• Statistical hypothesis testing is a useful
framework for many experimental
situations
• We will use a procedure known as the twosample t-test
31
The Hypothesis Testing Framework
• Sampling from a normal distribution
• Statistical hypotheses: H :   
0
1
2
H1 : 1   2
32
Estimation of Parameters
1 n
y   yi estimates the population mean 
n i 1
n
1
2
2
S 
( yi  y ) estimates the variance 

n  1 i 1
2
33
Summary Statistics (pg. 36)
Formulation 1
Formulation 2
“New recipe”
“Original recipe”
y1  16.76
y 2  17.04
S  0.100
S 22  0.061
S1  0.316
S 2  0.248
n1  10
n 2  10
2
1
34
How the Two-Sample t-Test Works:
Use the sample means to draw inferences about the population means
y1  y2  16.76  17.04  0.28
Difference in sample means
Standard deviation of the difference in sample means
 
2
y
2
n
This suggests a statistic:
Z0 
y1  y2
 12
n1

 22
n2
35
How the Two-Sample t-Test Works:
Use S and S to estimate  and 
2
1
2
2
2
1
The previous ratio becomes
2
2
y1  y2
2
1
2
2
S
S

n1 n2
However, we have the case where     
2
1
2
2
2
Pool the individual sample variances:
(n1  1) S  (n2  1) S
S 
n1  n2  2
2
p
2
1
2
2
36
How the Two-Sample t-Test Works:
The test statistic is
y1  y2
t0 
1 1
Sp

n1 n2
• Values of t0 that are near zero are consistent with the null
hypothesis
• Values of t0 that are very different from zero are consistent
with the alternative hypothesis
• t0 is a “distance” measure-how far apart the averages are
expressed in standard deviation units
• Notice the interpretation of t0 as a signal-to-noise ratio
37
The Two-Sample (Pooled) t-Test
(n1  1) S12  (n2  1) S22 9(0.100)  9(0.061)
S 

 0.081
n1  n2  2
10  10  2
2
p
S p  0.284
t0 
y1  y2
16.76  17.04

 2.20
1 1
1 1
Sp

0.284

n1 n2
10 10
The two sample means are a little over two standard deviations apart
Is this a "large" difference?
38
The Two-Sample (Pooled) t-Test
t0 = -2.20
• So far, we haven’t really
done any “statistics”
• We need an objective
basis for deciding how
large the test statistic t0
really is.
39
The Two-Sample (Pooled) t-Test
• A value of t0 between
–2.101 and 2.101 is
consistent with
equality of means
• It is possible for the
means to be equal and
t0 to exceed either
2.101 or –2.101, but it
would be a “rare
event” … leads to the
conclusion that the
means are different
• Could also use the
P-value approach
t0 = -2.20
40
Use of P-value in Hypothesis
testing
• P-value: smallest level of significance that
would lead to rejection of the null
hypothesis Ho
• It is customary to call the test statistic
significant when Ho is rejected. Therefore,
the P-value is the smallest level a at which
the data are significant.
41
Minitab Two-Sample t-Test Results
42
Checking Assumptions –
The Normal Probability Plot
• Assumptions
1.
2.
Equal variance
Normality
• Procedure:
1.
2.
3.
Rank the observations in the sample in an ascending order.
Plot ordered observations vs. observed cumulative frequency (j0.5)/n
If the plotted points deviate significantly from straight line, the
hypothesized model in not appropriate.
43
Checking Assumptions –
The Normal Probability Plot
44
Checking Assumptions –
The Normal Probability Plot
• The mean is estimated as the 50th percentile on the
probability plot.
• The standard deviation is estimated as the
differnce between the 84th and 50th percentiles.
• The assumption of equal population variances is
simply verified by comparing the slopes of the two
straight lines in F2-11.
45
Importance of the t-Test
• Provides an objective framework for simple
comparative experiments
• Could be used to test all relevant hypotheses
in a two-level factorial design.
46
Confidence Intervals (See pg. 43)
• Hypothesis testing gives an objective statement
concerning the difference in means, but it doesn’t
specify “how different” they are
• General form of a confidence interval
L    U where P( L    U )  1  a
• The 100(1- α)% confidence interval on the
difference in two means:
y1  y2  ta / 2,n1  n2 2 S p (1/ n1 )  (1/ n2 )  1  2 
y1  y2  ta / 2,n1  n2 2 S p (1/ n1 )  (1/ n2 )
47
Hypothesis testing
 12   22
• The test statitic becomes
to 
y1  y 2
S 12 S 22

n1 n 2
• This statistic is not distributed exactly as t.
• The distribution of to is well approximated2
2
2
S
/
n

S
by t if we use

1
1
2 / n2 
v 
2
2
2
2
as the DOF
S 1 / n1  S 2 / n 2 
n1  1

n2 1
48
Hypothesis
testing
2
2
 1 and  2 are known
• The test statitic becomes
zo 
y1  y 2

2
1
n1


2
2
n2
• If both populations are normal, or if the sample sizes are
large enough, the distribution of zo is N(0,1) if the null
hypothesis is true. Thus, the critical region would be found
using the normal distribution rather than the t.
• We would reject Ho, if z o  z a / 2 where za/2 is the upper
a/2 percentage point of the standard normal distribution
49
Hypothesis testing
 12 and  22 are known
• The 100(1-a) percent confidence interval:
y1  y 2  z a /2
 12
n1

 22
n2
 1  2  y 1  y 2  z a / 2
 12
n1

 22
n2
50
Hypothesis testing
Comparing a single mean to a specified value
• The hypothesises are: Ho:  = o and H1:  ≠ o
• If the population is normal with known variance, or if the
population is non-normal but the sample size is large
enough, then the hypothesis may be tested by direct
application of the normal distribution.
y  o
zo 
• Test statistic
/ n
• If Ho is true, then the distribution of zo is N(0,1).
Therefore, Ho is rejected if z  z
o
a /2
51
Hypothesis testing
Comparing a single mean to a specified value
The value of o is usually determined in one
of three ways:
1. From past evidence, knowledge, or
experimentation
2. The result of some theory or model describing
the situation under study
3. The result of contractual specifications
52
Hypothesis testing
Comparing a single mean to a specified value
• If the variance of the population is known, we assume that
the population is normally distributed.
• Test statistic t  y  o
o
S/ n
• Ho is rejected if t o  t a / 2,n 1
• The 100(1-a) percent confidence interval
y  t a / 2,n 1
S
S
   y  t a / 2,n 1
n
n
53
The paired comparison problem
The two tip hardness experiment
 i  1, 2
Statistical model y ij  i   j   ij 
j  1, 2,...,10

jth paired difference d j  y 1 j  y 2 j
•
•
• Expected value of paired difference d  1  2
• Testing hypothesis: Ho: d=0 and H1: d≠0
1/ 2
n


• Test statistic:
d d


j

d
1
j 1
to 
; d  d j ; S d  
n j 1
 n 1
Sd / n


n





54
The paired comparison problem
The two tip hardness experiment
• Randomized block design
• Block: homogenous experimental unit
• The block represents a restriction on complete
randomization because the treatment combinations are
only randomized within the block
Randomized Block
Complete Randomization
DOF
n-1 = 9
2n-2 = 18
Standard deviation
Sd = 1.2
Sp = 2.32
Confidence interval on -0.1±0.86
1-2
-0.1±2.18
55
Inferences about the variablity of
normal distributions
• Ho: 2 = o and H1: 2 ≠ o
n
2
• Test statistic
 y i  y 
c 
2
o
SS

i 1
 o2
 o2 2
• Ho is rejected if c 2  c 2
or co  c12a .2,n 1
o
a .2,n 1
• The 100(1-a) percent confidence interval
(n  1)S 2
ca2 / 2,n 1
2 
(n  1)S 2
c12a / 2,n 1
56
Inferences about the variablity
of normal distributions
H o :  12   22 and H 2 :  12   22
• Test statistic
S 12
Fo  2
S2
• Ho is rejected if Fo  Fa .2,n11,n 21 or Fo  F1a .2,n11,n 21
57