Download Unit 14: Nonparametric Statistical Methods

Unit 14: Nonparametric
Statistical Methods
Statistics 571: Statistical Methods
Ramón V. León
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
1
Introductory Remarks
• Most methods studied so far have been based on
the assumption of normally distributed data
– Frequently this assumption is not valid
– Sample size may be too small to verify it
• Sometimes the data is measured in an ordinal scale
• Nonparametric or distribution-free statistical
methods
– Make very few assumptions about the form of the
population distribution from which the data are sampled
– Based on ranks so they can be used on ordinal data
• Will concentrate on hypothesis tests but will also
mention confidence interval procedures.
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
2
Inference for a Single Sample
Consider a random sample x1 , x2 ,..., xn from a population
with unknown median µ .
(Recall that for nonnormal (especially skewed)
distributions the median is a better measure of the center
than the mean.)
H 0 : µ = µ0 vs. H1 : µ > µ0
Example: Test whether the median household income of a
population exceeds $50,000 based on a random sample of
household incomes from that population
For simplicity we sometimes present methods for one-sided tests.
Modifications for two-sided tests are straightforward and are given in the
textbook Some examples in these notes are two-sided tests.
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
3
Sign Test for a Single Sample
H 0 : µ = µ0 vs. H1 : µ > µ0
Sign test:
1. Count the number of xi 's that exceed µ0 . Denote this
number by s+ , called the number of plus signs. Let
s− = n − s+ , which is the number of minus signs.
2. Reject H 0 if s+ is large or equivalently if s− is small.
Test idea:
Under the null hypothesis s+ has a binomial distribution,
Bin (n, ½). So this test is simply the test for binomial proportions
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
4
Sign Test Example
A thermostat used in an electric device is to be checked for the
accuracy of its design setting of 200ºF. Ten thermostats were
tested to determine their actual settings, resulting in the
following data:
202.2, 203.4, 200.5, 202.5, 206.3, 198.0, 203.7, 200.8, 201.3, 199.0
H 0 : µ = 200 vs H1 : µ ≠ 200
s+ = 8 = number of data values > 200, so
10
10
2
10
10
  1 
  1 
P-value = 2∑     = 2∑     = 0.110
i =8  i   2 
i =0  i   2 
10
(The t test based on the mean has P-value = 0.0453. However recall that
the t test assumes a normal population)
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
5
Normal Approximation to Test Statistic
If the sample size is large ( ≥ 20) the common of S+ and S− is
approximated by a normal distribution with
1 n
E ( S + ) = E ( S− ) = np = n   = ,
2 2
 1  1  n
Var ( S+ ) = Var ( S − ) = np(1 − p) = n     =
 2  2  4
Therefore can perform a one-sided z - test with
s − n 2 −1 2
z= +
n4
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
6
P-values for Sign
Test Using JMP
Based on normal approximation
to the binomial ( = z2 )
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
7
Treatment of Ties
• Theory of the test assumes that the distribution of
the data is continuous so in theory ties are
impossible
• In practice they do occur because of rounding
• A simple solution is to ignore the ties and work
only with the untied observation. This does reduce
the effective sample size of the test and hence its
power, but the loss is not significant if there are
only a few ties
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
8
Comfidence Interval for µ
Let x(1) ≤ x(2) ≤ ⋅⋅⋅ ≤ x( n ) be the ordered data values.
Then a (1-α )-level CI for µ is given by
x(b +1) ≤ µ ≤ x( n −b )
where b = bn ,1−α 2 is the lower α 2 critical point
of the Bin ( n,1 2 ) distribution.
Note: Not all confidence levels are possible because
of the discreteness of the Binomial distribution
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
9
Thermostat Setting: Sign Confidence
Interval for the Median
From Table A.1 we see that for n = 10 and p=0.5,
the lower 0.011 critical point of the binomial
distribution is 1 and by symmetry the upper 0.011
critical point is 9.
Setting α 2 = 0.011 which gives 1-α = 1 − 0.022 = 0.978,
we find that
199.0 = x(2) ≤ µ ≤ x(9) = 203.7
is a 97.8% CI for µ .
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
10
Sign Test for Matched Pairs
Drop 3 tied pairs. Then s+ = 20; s- = 3
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
11
Sign Test for Matched Pairs
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
12
Sign Test for Matched Pairs in JMP
Pearson’s p-value is not the same
as the book’s two-sided P-value
because the book uses the
continuity correction in the
normal approximation to the
binomial distribution,
i.e, book uses z = 3.336 (Page
567) rather than z = 3.544745
used by JMP. Note that
(3.544745)2 = 12.5652
book
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
13
Wilcoxon Signed Rank Test
H 0 : µ = µ0 vs. H1 : µ ≠ µ0
More powerful than the sign test, however, it requires the
assumption that the population distribution is symmetric
1. Rank and order the differences in terms of their absolute value
Example 14.1 and 14.4: Thermostat Setting is 200° F
2. Calculate w+ = sum of the ranks of the positive differences
w+ = 6 + 8 + 1 + 7 + 10 + 9 + 2 + 4
3. Reject H0 if w+ is large or small
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
14
Wilcoxon Signed Rank Test in JMP
This test finds a
significant
difference at
α=0.05 while the
sign test did not
at even α=0.1
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
15
Normal Approximation in the Wilcoxon
Signed Rank Test
For large n, the null distribution of W ∼ W+ ∼ Wcan be well-approximated by a normal distribution
with mean and variance given by
n(n + 1)
n(n + 1)(2n + 1)
E (W ) =
and Var (W ) =
.
4
24
For large samples a one-sided (greater than median)
z-test uses the statistic
w+ − n(n + 1) / 4 − 1/ 2
z=
n(n + 1)(2n + 1) 24
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
16
Importance of Symmetric Population
Assumption
Here even though H0 is true the long right hand tail makes the positive
differences tend to be larger in magnitude than the negative differences,
resulting in higher ranks. This inflates w+ and hence the test’s type I error
probability.
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
17
Null
Distribution of
the Wilcoxon
Signed Rank
Statistics
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
18
Null Distribution of the
Wilcoxon Signed Rank Statistics
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
19
Wilcoxon Signed Rank Statistic:
Treatment of Ties
• There are two types of ties
– Some of the data is equal to the median
• Drop these observations
– Some of the differences from the median
may be tied
• Use midrank, that is, the average rank
For example, suppose
d1 = −1, d 2 = +3, d3 = −3, d 4 = +5
Then
r1 = 1, r2 = r3 =
7/26/2004
With ties Table A.10
is only approximate
(2 + 3)
= 2.5, r4 = 4
2
Unit 14 - Stat 571 - Ramón V. León
20
Wilcoxon Sign Rank Test:
Matched Pair Design
Example 14.5: Comparing Two
Methods of Cardiac Output
Notice that we drop the
three zero differences
Notice that we average the
tied ranks
Two-Side P-values
Signed test: 0.0008
Signed Rank test: 0.0002
t-test: 0.0000671 (Page 284)
(Notice that these tests require
progressively more stringent
assumptions about the
population of differences)
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
21
JMP Calculation
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
22
Signed Rank Confidence Interval for the Median
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
23
Thermostat Setting: Wilcoxon Signed
Rank Confidence Interval for Median
From Table A.10 we see that for n = 10, the upper 2.4%
critical point is 47 and by symmetry the lower 2.4%
10(10 + 1)
critical point is
- 47 = 55 - 47 = 8.
2
Setting α 2 = 0.024 and hence 1-α=1-0.048=0.952
we find that
+200.10 = x9 = x8+1 ≤ µ ≤ x47 = 203.55
is a 95.2% CI for µ
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
24
Inferences for Two Independent Samples
One wants to show that the observations from one population
tend to be larger than those from another population based on
independent random samples
x1 , x2 ,..., xn1 and y1 , y2 ,..., yn2
Examples:
•Treated patients tend to live longer than untreated patients
•An equity fund tends to have a higher yield than a bond fund
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
25
Wilcoxon-Mann-Whitney Test Example:
Time to Failure of Two Capacitor Groups
Reject for extreme values of w1.
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
26
Stochastic Ordering of Populations
X is stochastically larger than Y ( X Y )
if for all real numbers u ,
P ( X > u ) ≥ P(Y > u )
equivalently,
P ( X ≤ u ) = F1 (u ) ≤ F2 (u ) = P(Y ≤ u )
with strict inequality for at least some u.
Denoted by X Y or equivalently by F1 < F2 )
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
27
Stochastic Ordering Especial Case:
Location Difference
θ is called a location parameter
Notice that X 2 ≺ X 1 iff θ 2 < θ1
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
28
Wilcoxon-Mann-Whitney Test
H 0 : F1 = F2 ( X ∼ Y )
Alternatives :
One sided: H1 : F1 < F2 ( X
Y)
Two sided: H1 : F1 < F2 or F2 < F1 ( X
Y or Y
X)
Notice that the alternative is not H1 : F1 ≠ F2
(Kolmogorov-Smirnov Test can handle this alternative)
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
29
Wilcoxon Version of the Test
H 0 : F1 = F2 ( X ∼ Y ) vs. H1 : F1 < F2 ( X
Y)
1. Rank all N = n1 + n2 observations,
x1 , x2 ,..., xn1 and y1 , y2 ,..., yn2
in ascending order
2. Sum the ranks of the x's and y's separately.
Denote these sums by w1 and w2
3. Reject H 0 if w1 is large or equivalently w2 is small
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
30
Mann-Whitney Test Version
The advantage of using the Mann-Whitney form of the test is that
the same distribution applies whether we use u1 or u2
P − value = P(U ≥ u1 ) = P(U ≤ u2 )
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
31
Null Distribution of the WilcoxonMann-Whitney Test Statistic
Under the null hypothesis each of these 10 ordering has an
equal chance of occurring, namely, 1/10
5
  = 10
 2
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
32
Null Distribution of the WilcoxonMann-Whitney Test Statistic
P ( w1 ≥ 8) = 0.1 + 0.1 = 0.2 (one-sided p-value for w1 = 8)
7/26/2004
( H1 : X
Unit 14 - Stat 571 - Ramón V. León
Y)
33
Normal Approximation of MannWhitney Statistic
For large n1 and n2 , the null distribution of U can be
well approximated by a normal distribution with
mean and variance given by
n1n2
n1n2 ( N + 1)
and Var (U ) =
E (U ) =
2
12
A large sample one-sided z - test can be based on the statistic
u1 − n1n2 2 − 1 2
z=
n1n2 ( N + 1)
12
7/26/2004
( H1 : X
Unit 14 - Stat 571 - Ramón V. León
Y)
34
Treatment of Ties
A tie occurs when some x equal a y.
– A contribution of ½ is counted towards both u1
and u2 for each tied pair
– Equivalent to using the midrank method in
computing the Wilcoxon rank sum statistic
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
35
Wilcoxon-Mann-Whitney Confidence Interval
Example14.8 shows that [d(18) , d(63) ] = [-1.1, 14.7] is a 95.6% CI
for the difference of the two medians of the failure times of capacitors.
This example is in the book errata since Table A.11 is not detailed enough.
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
36
Wilcoxon-Mann-Whitney Test in JMP
With
continuity
correction.
Used in
the book
which gets
a onesided pvalue of
0.0502
Without
continuity
correction
7/26/2004
z2=1.6882
Unit 14 - Stat 571 - Ramón V. León
37
Inference for Several Independent Samples:
Kruskal-Wallis Test
Note that this
is a completely
randomized
design
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
38
Kruskal-Wallis Test
H 0 : F1 = F2 = ⋅⋅⋅ = Fa vs. H1 : Fi < Fj for some i ≠ j
Reject H 0 if kw > χ
7/26/2004
2
a −1,α
Distance from the
average rank
Unit 14 - Stat 571 - Ramón V. León
39
Chi-Square Approximation
• For large samples the distribution of KW under the
null hypothesis can be approximated by the chisquare distribution with a-1 degrees of freedom
• So reject H0 if
kw > χ a −1,α
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
40
Kruskal-Wallis Test Example
Reject if
kw is large.
2
χ 3,.005
= 12.837
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
41
Kruskal-Wallis Test in JMP
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
42
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
43
•Case method is different from Unitary method
•Formula method is different from Unitary method
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
44
Pairwise Comparisons:
Is Any Pair of Treatments Different?
•
One can use the Tukey Method on the average ranks to make
approximate pairwise comparisons.
•
This is one of many approximate techniques where ranks are
substituted for the observations in the normal theory methods.
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
45
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
46
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
47
Tukey’s Test Applied to the Ranks Averaged
Lack of
agreement
with the more
precise method
of Example
14.10. Here
Equation
method also
seems to be
different from
Formula and
Case method
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
48
Example of Friedman’s Test
Ranking is done within blocks
2
χ 7,.025
= 16.012 - P-value =.0040 vs. .0003 for ANOVA table
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
49
Inference for Several Matched Samples
Randomized Block Design:
i a ≥ 2 treatment groups
i b ≥ 2 blocks
i yij = observation on the i-th treatment in the j-th block
i Fij = c.d.f of r.v. Yij corresponding to the observed value yij
For simplicity assume Fij ( y ) = F ( y − θ i − β j )
iθ i is the "treatment effect"
i β j is the "block effect"
i.e., we assume that there is no
treatment by block interaction
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
50
Friedman Test
H 0 : θ1 = θ 2 = ⋅⋅⋅ = θ a vs. H1 : θ i > θ j for some i ≠ j
Reject if fr > χ
7/26/2004
2
a −1,α
Distance from the total
of the ranks from their
expected value when
there is no agreement
between the blocks
Unit 14 - Stat 571 - Ramón V. León
51
Pairwise Comparisons
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
52
Rank Correlation Methods
• The Pearson correlation coefficient
– measures only the degree of linear association between
two variables
– Inferences use the assumption of bivariate normality of
the two variables
• We present two correlation coefficients that
– Take into account only the ranks of the observations
– Measure the degree of monotonic (increasing or
decreasing) association between two variables
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
53
Motivating Example
( x, y ) = (1, e1 ), (2, e 2 ), (3, e3 ), (4, e 4 ), (5, e5 )
Note that there is a perfect positive association between between
x and y with y = ex.
•The Pearson correlation correlation coefficient is only 0.886
because the relationship is not linear
•The rank correlation coefficients we present yield a value of 1
for these data
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
54
Spearman’s Rank Correlation Coefficient
Ranges between –1 and +1 with rs = -1 when there is a perfect negative
association and rs = +1 when there is a perfect positive association
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
55
Example 14.12 (Wine Consumption and
Heart Disease Deaths per 100,000
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
56
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
57
Calculation of Spearman’s Rho
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
58
Test for Association Based on Spearman’s Rank
Correlation Coefficient
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
59
Hypothesis Testing Example
H 0 : X = Wine Consumption and
Y = Heart Disease Deaths are independent.
vs.
H1 : X and Y are (negatively or positively) associated
z = rS n − 1 = −0.826 19 − 1 = −3.504
Two-Sided P − value = 0.0004
Evidence of negative association
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
60
JMP Calculations:
Pearson Correlation
Heart Disease Deaths
350
300
250
200
150
100
50
0
2
4
6
8
10
Alcohol from Wine
Plot is fairly linear
Pearson correlation
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
61
JMP Calculations: Spearman Rank Correlation
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
62
Kendall’s Rank Correlation Coefficient:
Key Concept Examples
Concordant pairs:
(1,2), (4,9)
(1 - 4)(2 - 9)>0
(4,2), (3,1)
(4 - 3)(2 - 1)>0
Discordant pairs:
(1,2), (9,1)
(1 - 9)(2 - 1)<0
(2,4), (3,1)
(2 - 3)(4 - 1)<0
Tied pairs:
(1,3), (1,5)
(1,4), (2,4)
(1,2), (1,2)
7/26/2004
(1 – 1)(3 – 5)=0
(1 – 2)(4 – 4)=0
(1 – 1)(2 – 2)=0
Unit 14 - Stat 571 - Ramón V. León
Kendall’s
idea is to
compare the
number of
concordant
pairs to the
number of
discordant
pairs in
bivariate data
63
(X, Y)
(1, 2)
(3, 4)
(2, 1)
Kendall’s Tau
Example
n  3
Number of pairwise comparisons =   =   = 3 = N
 2   2
Concordant pairs:
(1,2) (3,4)
(3,4) (2,1)
Discordant pairs:
(1,2) (2,1)
Nc = 2
Nd=1
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
Nc − Nd
τˆ =
N
2 −1
=
3
1
=
3
64
Kendall’s Rank Correlation Coefficient:
Population Version
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
65
Kendall’s Rank Correlation Coefficient:
Sample Estimate
Let N c = Number of concordant pairs in the data
Let N d = Number of disconcordant pairs in the data
n
Let N =   be the number of pairwise comparisons among
2
the observations ( xi , yi ), i = 1, 2,..., n. Then
Nc − Nd
and N c + N d = N if no ties
N
Nc − Nd
ˆ
τ=
if ties
( N − Tx )( N − Ty )
τˆ =
where Tx and Ty are corrections for the number of tied pairs.
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
66
Hypothesis of Independence Versus Positive Association
Wine data:
-.696
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
-4.164
67
JMP
Calculations:
Kendall’s
Rank
Correlation
Coefficient
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
68
Kendall’s Coefficient of Concordance
• Measure of association between several matched samples
• Closely related to Friedman’s test statistic
– Consider a candidates (treatments) and b judges (blocks) with each
judge ranking the a candidates
• If there is perfect agreement between the judges, then each
candidate gets the same rank. Assuming the candidates are
labeled in the order of their ranking, the rank sum for the ith
candidate would be ri= ib
• If the judges rank the candidates completely at random
(“perfect disagreement”) then the expected rank of each
candidate would be [1+2+…+a]/a =[a(a+1)/2]/a=(a+1)/2,
and the expected value of all the rank sums would equal to
b(a+1)/2
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
69
Kendall’s Coefficient of Concordance
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
70
Kendall’s Coefficient of Concordance
and Friedman’s Test
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
71
w=
7/26/2004
24.667
= 0.881
4(8 − 1)
Unit 14 - Stat 571 - Ramón V. León
72
Do You Need to Know More
“Nonparametric Statistical Methods, Second Edition”
by Myles Hollander and Douglas A. Wolfe. (1999)
Wiley-Interscience
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
73
Resampling Methods
• Conventional methods are based on the sampling
distribution of a statistic computed for the
observed sample. The sampling distribution is
derived by considering all possible samples of size
n from the underlying population.
• Resampling methods generate the sampling
distribution of the statistic by drawing repeated
samples from the observed sample itself. This
eliminates the need to assume a specific functional
form for the population distribution (e.g. normal).
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
74
Challenger Shuttle O-Ring Data
Do we have statistical evidence that cold temperature leads to
more O-ring incidents?
•Notice that assumptions of two sample t test do not hold.
•Original analysis omitted the zeros? Was this justified?
•What do we do?
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
75
Wrong t-test Analysis
Difference of Low
mean to High mean
Notice that the assumptions of the
independent sample t-test do not hold,
i.e., data is not normal for each group.
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
76
Permutation Distribution of t Statistic
Also equal to the two-sided p-value
Equivalent to selecting all simple random samples without replacement of
size 20 from the 24 data points, labeling these High and the rest Low
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
77
Comments
• A randomization test is a permutation test applied
to data from a randomized experiment.
Randomization tests are the gold standard for
establishing causality.
• A permutation test considers all possible simple
random samples without replacement from the set
of observed data values
• The bootstrap method considers a large number of
simple random samples with replacement from
the set of observed data values.
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
78
Calculation of t Statistics from 10,000
Bootstrap Samples
-2
-1
7/26/2004
0
1
2
3
4
5
6
Think that we are placing the 24
Challenger data values in a hat.
And that we are randomly selecting
24 values with replacement from
the hat, labeling the first 20 values
High and the remaining 4 values
Low. We repeat these process
10,000 times. For each of these
10,000 bootstrap samples we
calculate the t-statistic. 35 tstatistics values were greater than
or equal to 3.888 out of 10000 (if sp
= 0, t is defined to be 0). This gives
a bootstrap P-value of 35/10000 =
0.0035
Unit 14 - Stat 571 - Ramón V. León
79
Bootstrap Distribution of Difference
Between the Means
67 of the 10,000 differences
of the Low mean and the
High mean were greater than
or equal to 1.3. This gives a
bootstrap P-value of
67/10000 = .0067
-1
7/26/2004
0
1
2
Conclusion: Cold
weather increases the
chance of O-ring
problems
Unit 14 - Stat 571 - Ramón V. León
80
Bootstrap Final Remarks
• The JMP files - that we used to generate the
bootstrap samples and to calculate the statistics are available at the course web site.
• There are bootstrap procedures for most types of
statistical problems. All are based on resampling
from the data.
• These methods do not assume specific functional
forms for the distribution of the data, e.g. normal
• The accuracy of bootstrap procedures depend on
the sample size and the number of bootstrap
samples generated
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
81
How Were
the
Bootstrap
Samples
Generated?
(see next page)
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
82
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
83
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
84
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
85
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
86
Calculated Columns in JMP Samples File
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
87
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
88
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
89
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
90
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
91
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
92
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
93
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
94
Bootstrap Estimate of the Standard Error of the Mean
Summary: We calculate the standard deviation of the N
bootstrap estimates of the mean
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
95
BSE for Arbitrary Statistic
Example: The bootstrap standard error of the median is calculated by
drawing a large number N, e.g. 10000, of bootstrap samples from the
data. For each bootstrap sample we calculated the sample median.
Then we calculate the standard deviation of the N bootstrap medians.
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
96
Estimated Bootstrap Standard Error for tstatistics Using JMP
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
Note N =10,000
97
Bootstrap Standard Error Interpretation
• Many bootstrap statistics have an
approximate normal distribution
• Confidence interval interpretation
– 68% of the time the bootstrap estimate (the
average of the bootstrap estimates) will be within
one standard error of true parameter value
– 95% of the time the bootstrap estimate (the
average of the bootstrap estimates) will be within
two standard error of true parameter value
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
98
Bootstrap Confidence Intervals –
Percentile Method: Median Example
1. Draw N (= 10000) bootstrap samples from the data and for
each calculate the (bootstrap) sample median.
2. The 2.5 percentile of the N bootstrap sample medians will
be the LCL for a 95% confidence interval
3. The 97.5 percentile of the N bootstrap sample medians will
be the UCL for a 95% confidence interval
0.025
0.025
LCL
7/26/2004
UCL
Unit 14 - Stat 571 - Ramón V. León
99
Do You Need to Know More?
“A Introduction to the Bootstrap”
by Bradley Efrom and Robert J. Tibshirani.
(1993) Chapman & Hall/CRC
7/26/2004
Unit 14 - Stat 571 - Ramón V. León
100