Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Nonsymmetric Bootstrap Confidence Intervals Using SASlIML® Software on a PC George W. Pasdirtz, Universtty 01 Wisconsin-Madison Abstract The Bootstrap In this paper, PC SAS/IML® modules are developed which calculate percentile method confidence intervals and vanants. The modules will work on distributions of parameters generated by any Monte Carlo technique, Including the bootstrap. Another approach, which has the benefit of being less costly, would be to use computer simulation to replicate the study. A Monte Carlo simulation would employ a random number generator to create experimental replications. The bootstrap variant uses the observed distribution of the data to generate the population distribution. Introduction As an example of the bootstrap algorithm, consider the general linear model where X and Yare independent and dependent variable matrices, respectively. In PC SAS/IML code, the coefficients can be estimated using The bootstrap is a ~eneral, computer-intensive resampling tech~ique for. testing hypotheses. It can be used with traditional estimators, such as ordinary least squares, or any other procedure that generates numerical estimates. It would be useful to have PROC IML modules which could be called to compute confidence intervals from any bootstrap algorithm that might be programmed in IML (for other approaches In SAS see Carson, 1985; Tibshirani, 1985; Westfall and Young,1989, and Westfall et. al. 1989). B = E = INV (X' *X) Y - * (X' *y) ; X*B; and the residuals saved in the matrix E. The study is then replicated &NBOOT-l times (where &NBOOT is a SAS macro variable), creating new observations by randomly drawing from the residuals. Acceptable confidence intervals are not available for all SAS® procedures. For example, both the maximum likelihood and weighted least squares estimates used in PROCs CATMOD, NUN and UFEREG produce standard errors (and thus confidence intervals) which are known to be bi.ased in. small samples (Freedman and Peters, 1984). Confidence Intervals for some nonparametric procedures In NPAR1WAYare not available (Lehmann, 1975; Bickel and Kneger, 1989). And, even though PROC GLM and PROC REG output can be used to construct confidence intervals the coverage probabilities may not be accurate outside the normal-theory paradigm. BV=SHAPE(B,NROW(B)#NCOL(B»: BSI1,lJ~BV' ; DO I~2 TO &NBOOT: INDEX=INT(NROW(Y) #UNIFORMIYI,lJII+l; BI=INV(X'*X)* (X*B+E [INDEX, 1: NCOL (Y) ] ) : BS [I, ] =SHAPE (BI, NROW (B) #NCOL (B) ) BMV=SHAPE(BM,NROW(BM)#NCOL(BM) ); COVB=COVB+(BV-BMV) * (BV-BMV) ' i END; The problems encountered in constructing a classical confidence interval can be seen from the the normal-theory form~I~. The true p~rameter, B \WhiCh can be a regression coefhclent, a median, a corre atlon coefficient etc.) is ' , assumed to be in an interval , ; An INDEX for sampling the residuals is created using the UNIFORM random number generator. Random numbers, on the interval [0,11, are multiplied by NROWIYI, truncated and rounded up (to avoid a zero index). The INDEX is then used in (X*B+E[INDEX,l:NCOL(Y)]) to generate a new set of Y observations from which new coefficients are est!mated and saved. Notice that saving the sample estimate of 8 as the first observation in the bootstrap vector, .BS, not only insures that it is represented in the population but also that probability estimates, such as (7) below, are accurate (see Noreen. 1989). (1 ) around the estimated coefficient. The constant za is a percentile point from a normal distribution. In order for the Interval to be accurate, the population from which the data are drawn must follow the distribution from which za is dr:3wn. The standard. error, 0", is. estimated from the sample. It IS SUbject to sampling fluctuation and biases introduced in the estlmallon of B. And, if the distribution of 8 is not symmetric, the ± operator is inappropriate; the coverage for the skewed tail of the distribution will be inaccurate. By comparison, a Monte Carlo simulation BI=INV(X"*X)*(X*B+NORMAL(Y)@SQRT(S2»; An atternative, brute force method, would be to develop the sampliry9. distribution of 8 directly: The actual study would be replicated a large number of times, say 200. With each replicallon, a new Biwould be estimated and stored in a ?AS data S?t. The distribution of 8 could be visually Inspected uSing PROC CHART with the HBAR command, which produces a horizontal bar chart with cumulative pe~centages.The upper 97.5 and the lower 2.5 percentage P~lnts would be read (with interpolation, if necessary) to give a 95%, tWO-Sided confidence interval. The interval ne?d not be symmetric and the upper and lower percentage points need not be the same. samples residuals, scaled lor the standard error of the regression, from a normal distribution. When the X and the Y data are correlated, the bootstrap estimates should be drawn from (x y) pairs (Tibshiranl 1985). " BI~INVIXIINDEX,l:NCOLIY)J' *X[INDEX,l:NCOL(Y)])*(Y[INDEX,l:NCOL(Y)]): In general, the bootstrap sample is asymptotically valid when drawn In the same manner as the original study. 1377 percentage points would be needed (SAS IML will truncate noninteger subscripts to the next lower value). The Bootstrap Standard Error One could alternatively construct a quantile estimator (Parrish and Carsel, 1988) A possible improvement on the normal theory confidence interval would be 13(1):5 ~2):5 ... :5 (2) p= Pr{p:<;;; 8} i=lnp+O.51 r=(np+O.5)- i 8= (1- r)p (i) + rp[i+l) which uses the bootstrap standard error. It is computed from .the diago~al elements of the bootstrap variancecovariance matnx S=SQRT(VECDIAG(CQVB/&NBOOT); (4) A remaining problem is that the percentile point must still be taken from a normal distribution. p(n) where the endpoints, IJ's, are computed from the order statistics, probability levels, and weighting factors. The SAS/IML module is Percentile Method START WQ (B, COL, PU, PL); N=NROW(B); PRINT "QUANTILE Since a vedor of B's is available, a natural approach would be to compute confidence intervals from the Inverse of the empirical distribution function (eelf). ESTIMATOR"; NP=N#-PU: T=SHAPE (0, N, 1); T[RANK(BI,COL»,)"B[,COL); BSTAR"T [: ]; I=INT(NP+O.5); R:::(NP+ 0.5)-1: + RnlI+l,l]; YU"(l-R)n[I,l] NP"N#PL; (3) I=INT(NP+0.5); PRINT "[" FINISH; where PU"1-&APLHA/2; The SAS IIML module START PRINT PCI (S,COL); "PERCENTILE METHOD UCI=SHAPE(O,l,l); .. i < LCI=UCI: A variant of the percentile method is the percentile-t confidence interval (Hall, 1986). Pi - 13' t. = --'c,--- LO=&ALPHA/2; HICI"T INUI,l) ; LOWCI"T[N#LO,l); PRINT n[" Lower I (5) "," BSTAR "," HICI (J boot "J": FINISH; where P' is the mean of the Pis. The estimated bootstrap parameters are first converted to I-statistics before applying (3). creates a sorted copy of one replicated parameter using T [RANK (B [, COL]) , J =8 [, COL J where COL is a given column of the bootstrap coefficient matrix. Then, the percentage points are computed using HI=1-&ALPHA/2: PL"&ALPHA/2; SAS PROC UNIVARIATE has a number of other methods for computing quantiles which have been evaluated by Parrish and Carsel (1988). T"SHAPE(O,NROW(B),l); TIRANK(BI,COL»,)"B[,COL); BSTAR"T I : ); N"NROW(B) ; HI=1-&ALPHA/2; R-(NP+0.5)-I; + R#T[I+1,l]; YL h,h BSTAR ",h YU "I"; YL"(l-R)tT[I,l] The edf relates the order statistics (the sorted bootstrap coefficients) to the cumulative probabil~ies. The edrl relates the cumulative probabilities to the original observations. Values for the edf and ~s inverse can be computed from the sorted values or read approximately from the output of PROC CHART with the HBAR option. Reflection Method To construct a confidence interval on an estimated bootstrap parameter, B', that has the same relationship to the true parameter, B, and an endpoint 6= edf- l (aI2), LO=&ALPHA/2: HlCl"T INUl, 1) ; LOWCI"T[N#LO,l); If ± [8- For example, the two-tailed 95% confidence interval is formed by taking the upper 97.5 and lower 2.5 percentage points of the distribution. If &NBOOT"200, then the lower percentage point (sorted in ascending order) is the fifth element, 200xO.025"5. f3] we would set B to the estimated value If bootstrap sample sizes are chosen which are even multiples of the desired confidence interval, an exact percentage point will always be found by this method. Otherwise, linear interpolation between observed 1378 IF and then compute the confidence interval (DIFF > T[KK,I]-T[K,I]) THEN DO; L=K; DIFF=T[KK,I]-T[K,I]; END: END; BSTAR=B[ :,COL]; LOW=T[L,I]; PRINT (6) "[" HI=T[L+LENGTH,I]; If II BSTAR HI LOW "]"; FINISH: The following SASlIML module uses the column average of the bootstrap coefficient matrix for the estimated parameter value. START REF Bias Corrected Percentile method (BCP) In 1982, Efron proposed a modification to the percentile method. He demonstrated that the percentile method is valid when the distribution of the bootstrap parameters is median unbiased, that is, (B, COL); N=NROW (B); PRINT BSTAR=B [:, COL] : "REFLECTION METHOD"; T=SHAPE (0, N, 1); T [RANK (B [, COL] ) , ] =B [, COL]; LIMITI=BSTARf2-T[Ni(I-&ALPHA/2) ,1]; LIMIT2=BSTARi2-T[Nf(&ALPHA/2) ,I]; PRINT n [n LIMIT! .. , .. BSTAR "," LIMIT2 .. J TO: sa}=o.50, FINISH; (7) An improvement on the reflection method is the calibrated confidence intelVal (Loh, t 987). If the coverage probability for either the empirical distribution function (The numerator is read as the number of bootstrap coefficients that are greater than or equal to the estimated sample value). The following SAS IML code DO J=I TO NROW (BMV) ; IF (BMV[J]**2 >= BV[J]**2) P[J]=P[J]+I; r or the probability distribution function a = 1- 2a is not precisely 95%, for example, one could fteratively change the probability level a until the exact coverage is obtained. l.oh (1987) suggests the following 'Y, = 'Y~'Y-;;' 'Y,='Yo+~-'YJ('Yo-'YJ~-'YJ -, END; could be used, wfth the estimator above, to accumulate a two-tailed probabilfty for each parameter estimate. When (7) is not true, Efron argues that there will be some transformation, usually unknown, which would normalize the bootstrap sampling distribution. Therefore, one can transform to and from the normal distribution, <1>, using 'Yn ~ 'Yo, 'Yn<Yo Zo = <I> -\edf( O}} which obtains the percentile point by simple linear interpolation between the actual and desired coverage probabilfties. The zero subscript indicates the desired and the n subscript indicates the actual coverage probabilities. A new probability level computed. a =(1- r,) / 2can THEN [edr\<I>(2Zo - Za}}] (8) then be <I>(z.,}=1-a to compute confidence intervals. Notice that if (7) is true then zerO, otherwise the bias correction, 2zo. is applied. Minimum Length Interval The SAS IML module (adapted from Buckland, 1985) Unlike a Monte Carlo simulation, the bootstrap simulation can only contain values that occur in the original sample. If START BCP{B,COL); N=NROW(B); PRINT "BIAS CORRECTED PERCENTILE METHOD"; P=SUM(B[,COL]<B[I, COL])/N + SUM(B[,COL]=B[I, COL])/(UN); ZED=PROBIT{P); PRINT "PROBABILITY" P; Z=-PROBIT(&ALPHA); LIMIT1=NiPROBNORM(ZED-Z) +0.5; LIMIT2=NiPROBNORM(ZED+Z) +0.5; T=SHAPE(O,N,l) ; we restrict ourselves to observed values and do not use interpolation, some pairing of values other than (3) might produce a confidence interval that has minimum length. The following SAS IML module (adapted from Buckland, 1985) searches among the order statistics to find the interval of minimum length. START LMIN (B, COL) ; N=NROW (B) ; PRINT "MINIMUM LENGTH INTERVAL" ; T[RANK(B[,COL]),]=B[,COL]; LIMITI=T[NtLIMITI/IOO,I]; LIMIT2=T[NfLIMIT2/IOO,I]; BSTAR=T[:]; PRINT "[" LIMITl TO," BSTAR to," LIMITL=&ALPHA/2i(N+1) +0.5: LIMITU=(1-(&ALPHA/2»f(N+I)+O.5; LENGTH=LIMITU-LIMITL; NUMB=N~LENGTH; T=SHAPE (0, N,I); T [RANK(B [, COL]) , ] =B [,COL] ; DIFF=T(N,l]-T[l,l]; DO K=l TO NUMB: LIMIT2 .. ] "; FINISH; L=O; KK=K+LENGTH: 1379 computes the BCP intervals. Notice that endpoint probabilities are splft P~SUM(BLCOLI + <B[l, Carson, R. T. (1985), "SAS® Macros for Bootstrapping and Cross-Validating Regression Equations," SUGI 10 Proceedings, 1064-1069. COLI) /N SUM(B[,COLI~B[l, COLI)/(UN); DiCiccio, T. and R.Tibshirani (1987), "Bootstrap Confidence Intervals and Bootstrap Approximations,· JASA, 82, 163-170. based on arguments by Efron (1982). A more general version of the BCP method, called the Accelerated Bias Corrected Percentile Method was proposed by Efron in 1987. The BCPA approach also takes the SKEW of the distribution a= SKEW = -1-i[ 6 Xj - 6Nj~1 (J Ed9ington, E. S. (1987), Randomization Tests, Second Edition, New York: Marcel Dekker. xJ Efron, B. (1979), "Bootstrap Methods: Another Look at the The Annals of Stalistics, 7, 1-26. Jackkn~e," (1981), "Censored Data and the Bootstrap," JASA, 76,312-319. into account before applying the transformation _ _ (1982), The Jackknife, the Bootstrap and Other Resampling Plans, Philadelphia: SIAM. (1987), "Better Bootstrap Confidence Intervals," (WiiIidiscussion) JASA, 82, 171-200. (7) _ _ and R. Tibshirani (1986), "Bootstrap methods for standard errors, confidence intervals and other measures of statistical accuracy," Statistical Science, 1,54-75. Notice that if the SKEW is zero, the BCPA method reverts to the BCP transformation. Freedman, D. A. (1981), "Bootstrapping Regression Models," The Annals of Statistics, 9, 1218-1228. Using the Modules and S. C. Peters (1984), "Bootstrapping a Regression Equation: Some Empirical Results; JASA, 79, 97-106. A number of questions can be anticipated: Hall, P. (1986), "On the Number of Bootstrap Simulations Required to Construct a Confidence Interval," The Annals of Statistics, 14, 1453-1462. First, is the bootstrap valid? Theoretical proofs are only available for a few well defined, yet general situations such as the ordinary least squares estimator (see Freedman, 1981). A useful simulation exercise would be to compare Monte Carlo and bootstrap estimators using the code above (but see Loh, 1985, for some warnings about Monte Carlo sImulations). (1988), "Theoretical Comparison of Bootstrap Coiilldence Intervals," (with discussion)The Annals of Statistics, 16,927-985. Lehmann, E. L. (1975), Nonparametrics: Statistical Techniques Based on Ranks. San Francisco: Holden-Day. Second, which confidence interval module is the best? Since the SCPA module makes the fewest assumptions, it is believed to be most general. However, h may not be appropnate where the underlying distribution is discrete. Where interpolation is acceptable, a comparative simulation study following Parrish and Carsel (1988) would be very interesting. Measures of bias, mean squared error and coverage could be compute and evaluated. Loh, W. (1985), "A New Method for Testing Separate Families 01 Hypotheses," JASA, 80, 362-368. _ _ (1987), "Calibrating Confidence Coefficients," JASA, 82, 155-162. Noreen, E. W. (1989), Computer Intensive Methods for Testing Hypotheses, New York: Wiley. Third, how many simulations are necessary to compute accurate confidence intervals? Currently, the consensus is that many more replications are needed for confidence intervals than for estimation of standard errors. Efron and Tibshirani (1986) suggest 100 fof standard errors and 1000 for confidence "'tervals. Buckland (1985) suggests that Parrish, R. S. and R. F. Carsel (t 988), "Comparison of Quantile Estimation Methods in the Univariate Procedure," SUGI13Proceedings,1011-t016. confidence intervals are not valid until at least five observations are in the appropriate tail of the distribution. For a large number of parameters, the routines as written may run out of memory, particularly on a PC without expanded memory. Schenker, N. (1985), "Qualms About Bootstrap Confidence Intervals," JASA, 80, 360-361. Tibshirani, R. J. (1985), "Bootstrap Computations," SUGI 10 Proceedings, 1059-1063. Fourth, are the modules ready for production use? No. Various input checks and checks on the validity of calculations should be added to the code (see Buckland, 1985). Westfall, P. H. and S. S. Young (1989) "P-value Adjustments for Multiple Tests in Multivariate Binomial Models," SUGI14 Proceedings, 1381-1384. ~ Y. Lin, and S. S. Young (1989), "A Procedure for the Analysis of Muttivariate Binomial Data with Adjustments for Muttlplichy," SUGI14 Proceedings, 1385-1392. References Bickel, P. J. and A. M. Krieger (1989), "Confidence Bands for a Distribution Function Using the Bootstrap," JASA, 84, 95-100. SAS AND SAS/IML are registered trademarks of the SAS Institute, Inc., Cal)', NC, USA. Buckland, S. T. (1985), "Calculation of Monte Carlo Confidence Intervals; Applied Statistics 34, 296-301. 1380