Download Nonsymmetric Bootstrap Confidence Intervals Using SAS/IML® Software on a PC

Nonsymmetric Bootstrap Confidence Intervals
Using SASlIML® Software on a PC
George W. Pasdirtz, Universtty 01 Wisconsin-Madison
Abstract
The Bootstrap
In this paper, PC SAS/IML® modules are developed which
calculate percentile method confidence intervals and
vanants. The modules will work on distributions of
parameters generated by any Monte Carlo technique,
Including the bootstrap.
Another approach, which has the benefit of being less
costly, would be to use computer simulation to replicate the
study. A Monte Carlo simulation would employ a random
number generator to create experimental replications. The
bootstrap variant uses the observed distribution of the data
to generate the population distribution.
Introduction
As an example of the bootstrap algorithm, consider the
general linear model where X and Yare independent and
dependent variable matrices, respectively. In PC SAS/IML
code, the coefficients can be estimated using
The bootstrap is a ~eneral, computer-intensive resampling
tech~ique for. testing hypotheses. It can be used with
traditional estimators, such as ordinary least squares, or
any other procedure that generates numerical estimates. It
would be useful to have PROC IML modules which could
be called to compute confidence intervals from any
bootstrap algorithm that might be programmed in IML (for
other approaches In SAS see Carson, 1985; Tibshirani,
1985; Westfall and Young,1989, and Westfall et. al. 1989).
B
=
E
=
INV (X' *X)
Y -
* (X' *y)
;
X*B;
and the residuals saved in the matrix E. The study is then
replicated &NBOOT-l times (where &NBOOT is a SAS macro
variable), creating new observations by randomly drawing
from the residuals.
Acceptable confidence intervals are not available for all
SAS® procedures. For example, both the maximum
likelihood and weighted least squares estimates used in
PROCs CATMOD, NUN and UFEREG produce standard
errors (and thus confidence intervals) which are known to
be bi.ased in. small samples (Freedman and Peters, 1984).
Confidence Intervals for some nonparametric procedures In
NPAR1WAYare not available (Lehmann, 1975; Bickel and
Kneger, 1989). And, even though PROC GLM and PROC
REG output can be used to construct confidence intervals
the coverage probabilities may not be accurate outside the
normal-theory paradigm.
BV=SHAPE(B,NROW(B)#NCOL(B»:
BSI1,lJ~BV' ;
DO I~2 TO &NBOOT:
INDEX=INT(NROW(Y)
#UNIFORMIYI,lJII+l;
BI=INV(X'*X)*
(X*B+E [INDEX, 1: NCOL (Y) ] ) :
BS [I, ] =SHAPE (BI, NROW (B) #NCOL (B) )
BMV=SHAPE(BM,NROW(BM)#NCOL(BM) );
COVB=COVB+(BV-BMV) * (BV-BMV) ' i
END;
The problems encountered in constructing a classical
confidence interval can be seen from the the normal-theory
form~I~. The true p~rameter, B \WhiCh can be a regression
coefhclent, a median, a corre atlon coefficient etc.) is
'
,
assumed to be in an interval
, ;
An INDEX for sampling the residuals is created using the
UNIFORM random number generator. Random numbers,
on the interval [0,11, are multiplied by NROWIYI, truncated
and rounded up (to avoid a zero index). The INDEX is then
used in (X*B+E[INDEX,l:NCOL(Y)]) to generate a new
set of Y observations from which new coefficients are
est!mated and saved. Notice that saving the sample
estimate of 8 as the first observation in the bootstrap
vector, .BS, not only insures that it is represented in the
population but also that probability estimates, such as (7)
below, are accurate (see Noreen. 1989).
(1 )
around the estimated coefficient. The constant za is a
percentile point from a normal distribution. In order for the
Interval to be accurate, the population from which the data
are drawn must follow the distribution from which za is
dr:3wn. The standard. error, 0", is. estimated from the sample.
It IS SUbject to sampling fluctuation and biases introduced in
the estlmallon of B. And, if the distribution of 8 is not
symmetric, the ± operator is inappropriate; the coverage for
the skewed tail of the distribution will be inaccurate.
By comparison, a Monte Carlo simulation
BI=INV(X"*X)*(X*B+NORMAL(Y)@SQRT(S2»;
An atternative, brute force method, would be to develop the
sampliry9. distribution of 8 directly: The actual study would
be replicated a large number of times, say 200. With each
replicallon, a new Biwould be estimated and stored in a
?AS data S?t. The distribution of 8 could be visually
Inspected uSing PROC CHART with the HBAR command,
which produces a horizontal bar chart with cumulative
pe~centages.The upper 97.5 and the lower 2.5 percentage
P~lnts would be read (with interpolation, if necessary) to
give a 95%, tWO-Sided confidence interval. The interval
ne?d not be symmetric and the upper and lower percentage
points need not be the same.
samples residuals, scaled lor the standard error of the
regression, from a normal distribution.
When the X and the Y data are correlated, the bootstrap
estimates should be drawn from (x y) pairs (Tibshiranl
1985).
"
BI~INVIXIINDEX,l:NCOLIY)J'
*X[INDEX,l:NCOL(Y)])*(Y[INDEX,l:NCOL(Y)]):
In general, the bootstrap sample is asymptotically valid
when drawn In the same manner as the original study.
1377
percentage points would be needed (SAS IML will truncate
noninteger subscripts to the next lower value).
The Bootstrap Standard Error
One could alternatively construct a quantile estimator
(Parrish and Carsel, 1988)
A possible improvement on the normal theory confidence
interval would be
13(1):5 ~2):5 ... :5
(2)
p= Pr{p:<;;; 8}
i=lnp+O.51
r=(np+O.5)- i
8= (1- r)p (i) + rp[i+l)
which uses the bootstrap standard error. It is computed
from .the diago~al elements of the bootstrap variancecovariance matnx
S=SQRT(VECDIAG(CQVB/&NBOOT);
(4)
A remaining problem is that the percentile point must still
be taken from a normal distribution.
p(n)
where the endpoints, IJ's, are computed from the order
statistics, probability levels, and weighting factors. The
SAS/IML module is
Percentile Method
START WQ (B, COL, PU, PL);
N=NROW(B); PRINT "QUANTILE
Since a vedor of B's is available, a natural approach would
be to compute confidence intervals from the Inverse of the
empirical distribution function (eelf).
ESTIMATOR";
NP=N#-PU:
T=SHAPE (0, N, 1);
T[RANK(BI,COL»,)"B[,COL);
BSTAR"T [: ];
I=INT(NP+O.5);
R:::(NP+ 0.5)-1:
+ RnlI+l,l];
YU"(l-R)n[I,l]
NP"N#PL;
(3)
I=INT(NP+0.5);
PRINT "["
FINISH;
where
PU"1-&APLHA/2;
The SAS IIML module
START
PRINT
PCI (S,COL);
"PERCENTILE METHOD
UCI=SHAPE(O,l,l);
.. i
<
LCI=UCI:
A variant of the percentile method is the percentile-t
confidence interval (Hall, 1986).
Pi - 13'
t. = --'c,---
LO=&ALPHA/2;
HICI"T INUI,l) ;
LOWCI"T[N#LO,l);
PRINT n["
Lower
I
(5)
","
BSTAR "," HICI
(J boot
"J":
FINISH;
where P' is the mean of the Pis. The estimated bootstrap
parameters are first converted to I-statistics before applying
(3).
creates a sorted copy of one replicated parameter using
T [RANK (B [, COL]) , J =8 [, COL J
where COL is a given
column of the bootstrap coefficient matrix. Then, the
percentage points are computed using
HI=1-&ALPHA/2:
PL"&ALPHA/2;
SAS PROC UNIVARIATE has a number of other methods
for computing quantiles which have been evaluated by
Parrish and Carsel (1988).
T"SHAPE(O,NROW(B),l);
TIRANK(BI,COL»,)"B[,COL);
BSTAR"T I : );
N"NROW(B) ;
HI=1-&ALPHA/2;
R-(NP+0.5)-I;
+ R#T[I+1,l];
YL h,h BSTAR ",h YU "I";
YL"(l-R)tT[I,l]
The edf relates the order statistics (the sorted bootstrap
coefficients) to the cumulative probabil~ies. The edrl
relates the cumulative probabilities to the original
observations. Values for the edf and ~s inverse can be
computed from the sorted values or read approximately
from the output of PROC CHART with the HBAR option.
Reflection Method
To construct a confidence interval on an estimated
bootstrap parameter, B', that has the same relationship to
the true parameter, B, and an endpoint 6= edf- l (aI2),
LO=&ALPHA/2:
HlCl"T INUl, 1) ;
LOWCI"T[N#LO,l);
If ± [8-
For example, the two-tailed 95% confidence interval is
formed by taking the upper 97.5 and lower 2.5 percentage
points of the distribution. If &NBOOT"200, then the lower
percentage point (sorted in ascending order) is the fifth
element, 200xO.025"5.
f3]
we would set B to the estimated value
If bootstrap sample sizes are chosen which are even
multiples of the desired confidence interval, an exact
percentage point will always be found by this method.
Otherwise, linear interpolation between observed
1378
IF
and then compute the confidence interval
(DIFF > T[KK,I]-T[K,I]) THEN DO;
L=K; DIFF=T[KK,I]-T[K,I];
END:
END;
BSTAR=B[ :,COL];
LOW=T[L,I];
PRINT
(6)
"["
HI=T[L+LENGTH,I];
If
II
BSTAR
HI
LOW
"]";
FINISH:
The following SASlIML module uses the column average of
the bootstrap coefficient matrix for the estimated parameter
value.
START REF
Bias Corrected Percentile method (BCP)
In 1982, Efron proposed a modification to the percentile
method. He demonstrated that the percentile method is
valid when the distribution of the bootstrap parameters is
median unbiased, that is,
(B, COL);
N=NROW (B); PRINT
BSTAR=B [:, COL] :
"REFLECTION
METHOD";
T=SHAPE (0, N, 1); T [RANK (B [, COL] ) , ] =B [, COL];
LIMITI=BSTARf2-T[Ni(I-&ALPHA/2) ,1];
LIMIT2=BSTARi2-T[Nf(&ALPHA/2) ,I];
PRINT n [n LIMIT! .. , .. BSTAR "," LIMIT2 .. J TO:
sa}=o.50,
FINISH;
(7)
An improvement on the reflection method is the calibrated
confidence intelVal (Loh, t 987). If the coverage probability
for either the empirical distribution function
(The numerator is read as the number of bootstrap
coefficients that are greater than or equal to the estimated
sample value). The following SAS IML code
DO J=I TO NROW (BMV) ;
IF (BMV[J]**2 >= BV[J]**2)
P[J]=P[J]+I;
r
or the probability distribution function
a = 1- 2a is not
precisely 95%, for example, one could fteratively change
the probability level a until the exact coverage is obtained.
l.oh (1987) suggests the following
'Y, = 'Y~'Y-;;'
'Y,='Yo+~-'YJ('Yo-'YJ~-'YJ
-,
END;
could be used, wfth the estimator above, to accumulate a
two-tailed probabilfty for each parameter estimate.
When (7) is not true, Efron argues that there will be some
transformation, usually unknown, which would normalize
the bootstrap sampling distribution. Therefore, one can
transform to and from the normal distribution, <1>, using
'Yn ~ 'Yo,
'Yn<Yo
Zo = <I> -\edf( O}}
which obtains the percentile point by simple linear
interpolation between the actual and desired coverage
probabilfties. The zero subscript indicates the desired and
the n subscript indicates the actual coverage probabilities.
A new probability level
computed.
a =(1- r,) / 2can
THEN
[edr\<I>(2Zo - Za}}]
(8)
then be
<I>(z.,}=1-a
to compute confidence intervals. Notice that if (7) is true
then zerO, otherwise the bias correction, 2zo. is applied.
Minimum Length Interval
The SAS IML module (adapted from Buckland, 1985)
Unlike a Monte Carlo simulation, the bootstrap simulation
can only contain values that occur in the original sample. If
START BCP{B,COL);
N=NROW(B);
PRINT
"BIAS
CORRECTED
PERCENTILE METHOD";
P=SUM(B[,COL]<B[I, COL])/N
+ SUM(B[,COL]=B[I, COL])/(UN);
ZED=PROBIT{P); PRINT "PROBABILITY" P;
Z=-PROBIT(&ALPHA);
LIMIT1=NiPROBNORM(ZED-Z) +0.5;
LIMIT2=NiPROBNORM(ZED+Z) +0.5;
T=SHAPE(O,N,l) ;
we restrict ourselves to observed values and do not use
interpolation, some pairing of values other than (3) might
produce a confidence interval that has minimum length.
The following SAS IML module (adapted from Buckland,
1985) searches among the order statistics to find the
interval of minimum length.
START LMIN (B, COL) ;
N=NROW (B) ;
PRINT
"MINIMUM
LENGTH
INTERVAL" ;
T[RANK(B[,COL]),]=B[,COL];
LIMITI=T[NtLIMITI/IOO,I];
LIMIT2=T[NfLIMIT2/IOO,I];
BSTAR=T[:];
PRINT "[" LIMITl TO," BSTAR to,"
LIMITL=&ALPHA/2i(N+1) +0.5:
LIMITU=(1-(&ALPHA/2»f(N+I)+O.5;
LENGTH=LIMITU-LIMITL;
NUMB=N~LENGTH;
T=SHAPE (0, N,I);
T [RANK(B [, COL]) , ] =B [,COL] ;
DIFF=T(N,l]-T[l,l];
DO K=l TO NUMB:
LIMIT2
.. ] ";
FINISH;
L=O;
KK=K+LENGTH:
1379
computes the BCP intervals. Notice that endpoint
probabilities are splft
P~SUM(BLCOLI
+
<B[l,
Carson, R. T. (1985), "SAS® Macros for Bootstrapping and
Cross-Validating Regression Equations," SUGI 10
Proceedings, 1064-1069.
COLI) /N
SUM(B[,COLI~B[l,
COLI)/(UN);
DiCiccio, T. and R.Tibshirani (1987), "Bootstrap
Confidence Intervals and Bootstrap Approximations,·
JASA, 82, 163-170.
based on arguments by Efron (1982).
A more general version of the BCP method, called the
Accelerated Bias Corrected Percentile Method was
proposed by Efron in 1987. The BCPA approach also takes
the SKEW of the distribution
a=
SKEW
= -1-i[
6
Xj -
6Nj~1
(J
Ed9ington, E. S. (1987), Randomization Tests, Second
Edition, New York: Marcel Dekker.
xJ
Efron, B. (1979), "Bootstrap Methods: Another Look at the
The Annals of Stalistics, 7, 1-26.
Jackkn~e,"
(1981), "Censored Data and the Bootstrap," JASA,
76,312-319.
into account before applying the transformation
_ _ (1982), The Jackknife, the Bootstrap and Other
Resampling Plans, Philadelphia: SIAM.
(1987), "Better Bootstrap Confidence Intervals,"
(WiiIidiscussion) JASA, 82, 171-200.
(7)
_ _ and R. Tibshirani (1986), "Bootstrap methods for
standard errors, confidence intervals and other measures
of statistical accuracy," Statistical Science, 1,54-75.
Notice that if the SKEW is zero, the BCPA method reverts
to the BCP transformation.
Freedman, D. A. (1981), "Bootstrapping Regression
Models," The Annals of Statistics, 9, 1218-1228.
Using the Modules
and S. C. Peters (1984), "Bootstrapping a
Regression Equation: Some Empirical Results; JASA, 79,
97-106.
A number of questions can be anticipated:
Hall, P. (1986), "On the Number of Bootstrap Simulations
Required to Construct a Confidence Interval," The Annals
of Statistics, 14, 1453-1462.
First, is the bootstrap valid? Theoretical proofs are only
available for a few well defined, yet general situations such
as the ordinary least squares estimator (see Freedman,
1981). A useful simulation exercise would be to compare
Monte Carlo and bootstrap estimators using the code
above (but see Loh, 1985, for some warnings about Monte
Carlo sImulations).
(1988), "Theoretical Comparison of Bootstrap
Coiilldence Intervals," (with discussion)The Annals of
Statistics, 16,927-985.
Lehmann, E. L. (1975), Nonparametrics: Statistical
Techniques Based on Ranks. San Francisco: Holden-Day.
Second, which confidence interval module is the best?
Since the SCPA module makes the fewest assumptions, it
is believed to be most general. However, h may not be
appropnate where the underlying distribution is discrete.
Where interpolation is acceptable, a comparative
simulation study following Parrish and Carsel (1988) would
be very interesting. Measures of bias, mean squared error
and coverage could be compute and evaluated.
Loh, W. (1985), "A New Method for Testing Separate
Families 01 Hypotheses," JASA, 80, 362-368.
_ _ (1987), "Calibrating Confidence Coefficients," JASA,
82, 155-162.
Noreen, E. W. (1989), Computer Intensive Methods for
Testing Hypotheses, New York: Wiley.
Third, how many simulations are necessary to compute
accurate confidence intervals? Currently, the consensus is
that many more replications are needed for confidence
intervals than for estimation of standard errors. Efron and
Tibshirani (1986) suggest 100 fof standard errors and 1000
for confidence "'tervals. Buckland (1985) suggests that
Parrish, R. S. and R. F. Carsel (t 988), "Comparison of
Quantile Estimation Methods in the Univariate Procedure,"
SUGI13Proceedings,1011-t016.
confidence intervals are not valid until at least five
observations are in the appropriate tail of the distribution.
For a large number of parameters, the routines as written
may run out of memory, particularly on a PC without
expanded memory.
Schenker, N. (1985), "Qualms About Bootstrap Confidence
Intervals," JASA, 80, 360-361.
Tibshirani, R. J. (1985), "Bootstrap Computations," SUGI
10 Proceedings, 1059-1063.
Fourth, are the modules ready for production use? No.
Various input checks and checks on the validity of
calculations should be added to the code (see Buckland,
1985).
Westfall, P. H. and S. S. Young (1989) "P-value
Adjustments for Multiple Tests in Multivariate Binomial
Models," SUGI14 Proceedings, 1381-1384.
~
Y. Lin, and S. S. Young (1989), "A Procedure for the
Analysis of Muttivariate Binomial Data with Adjustments for
Muttlplichy," SUGI14 Proceedings, 1385-1392.
References
Bickel, P. J. and A. M. Krieger (1989), "Confidence Bands
for a Distribution Function Using the Bootstrap," JASA, 84,
95-100.
SAS AND SAS/IML are registered trademarks of the SAS
Institute, Inc., Cal)', NC, USA.
Buckland, S. T. (1985), "Calculation of Monte Carlo
Confidence Intervals; Applied Statistics 34, 296-301.
1380