Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
APPLICATIONS OF THE IML PROCEDURE FOR MULTIPLE RESPONSE PERMUTATION TESTS William D. Johnson and Donald E. Mercante William D. Johnson, Lonisiana State University Medical Center New Orleans, LA 70112-1393 Keywords :PROC IML, Multivariate Analysis, Multiresponse Permutation Procedures, Clinical Trials adjusted p-values may be better than their unadjusted counterparts, they are not suitable alternatives to a p-value based on an appropriate global test. Abstract The usual parametric statistics require two related sets of assumptions regarding the data. First, the data for each subject must be measured on a common scale at least On the ordinal level. Second, the data must satisfy the distributional assumptions Of the statistical technique. Typically, the data must be normally distributed or amenable to a transformation which will make the data normally distributed. This article discusses statistical methods for the analysis of multivariate data in experiments with a relatively small number of subjects. Methods based on the assumption of multivariate normality have power that decreases rapidly as the number of variates increases for a fixed sample size. In some investigations, the number of variates exceeds the sample size creating an awkward dilemma in classical multivariate analysis. Permutation tests have many useful applications especiaJly where the assumptions such as normal distributions and hornescedasticity are in question. Additionally, the number of variates relative to the sample size is not an issue with these tests .. Permutation tests based' on randomization theory provide an alternative class of tests that are often overlooked in multivariate analysis. Although permutation tests are well known and frequently used in the analysis of univariate data, they are used infrequently in multivariate analysis presumably either because the procedures are not well known or computer programs are not readily available for the computations. Mielke and his colleagues described a multiresponse permutation procedure and demonstrated its use in a variety of applications especially in meteorological investigations [1-7]. The test is exact but may require an inordinate amount of computing time for a large number. of permutations. Mielke [1] developed an accurate approximation to the exact permutation test based on the beta distribution that is appropriate for large sample sizes. Our experience has been that the approximation yields p-values very close to those of the exact test even for modest sample sizes and the computer time required is negligible. We discuss a multivariate permutation test and present a readily implemented IML procedure for constructing these tests. Software for the multiple response permutation tests was available previously in the SAS® Supplemental Library as PROC MRPP but has been discontinued in SAS Version 6. Our IML program in, SAS Version 6 fills this void as illustrated in the three examples presented. Introduction Clinical trials and other experiments often involve a small number of subjects and a relatively large number of observations on each subject; In some instances, the multivariate data structure involves a single variate observed repeatedly either under different experimental conditions and/or longitudinally over time. The analysis is often approached through a repeated measures or split-plot analysis of variance in these situations. When a number of different types of variates must be dealt with in the analysis and the sample size is small, many analysts focus on one variate at a time and perform separate univariate analyses. Some analysts attempt to adjust the resulting p-values for multiple testing, but while The method is based on the concept of multidimensional distances between subjects and utilizes permutation tests to make group comparisons. The response variates must be measured on at least an ordinal scale. For a given set ofdata, the analyst can choose a metric from a family of distance functions. In most applications, either Euclidean or squared Euclidean distance is an appropriate choice. To carry out tests of hypotheses, a permutation procedure can be used to determine the exact p-value or it can be estimated by the approximate procedure. 1027 Alternatively, one could use the pennutation procedure to estimate the p-value by evaluating a random sample of all possiblepennlltations. An exact probability requires the complete enumeration ·of all pennutations of the n subjects considered nJ, n2, ... , ng at a time. There are M = n!(nl!,n2!, ... ,ng!)-1 such pennutations. The test statistic d is calculated for each of the M pennutations. Under the null hypothesis, each of the M pennutations has probability ~ of occurrence, the exact p-value is detennined as the proportion of ~ values less than or equal to the realized (observed) d value. In practice, the number of pennutations can quickly become fonnidable even for moderate sample sizes. If the maximum number of pennutations to be perfonned is large, we switch to random sampling of all possible pennutations to detenniDe the significance of the observed data or implement an approximate test based on the beta distributioil. For brevity we do not discuss the approximate test here but rather refer the reader to Mielke [I]. The purpose of this article is to describe pennutation tests for the analysis of small sample multivariate data sets; The next section gives a mathematical description of the multivariate pennuta\ion test. The subsequent section provides a brief description of the statements required to execute the program. Its use is further illustrated with examples in later sections. Multivariate Permutation Test The test criterion is based on a weighted group average ofa mean between subject distance function for all subjects within a group. The distance function may be based on the raw data or linear functions of the raw data. The linear functions are used to generate data structures for testing certain hypotheses of interest such as parallelism for longitudinal data. In this context, the hypothesis to be tested is H. : Switching subjects among groups should have no effect on the averag~ within and among group distances, and the alternative is HA : Switching subjects among groups increases the average within group distances and decreases the average among group distance. Let g denote the number of groups to be compared in a completely randomized design with nJ,n,. ... ,ng subjects in each group where n,,, 2; and n = ~n;. Let y~ = {YI]'Y21, ••• ,YpI} denote p commensurate observations on subject 1. Let each· pair of pdimensional response vectors be summarized by a symmetric distance function such as dIJ= d; = ( n; 2 ) -I . To illustrate the use of the program consider the data in Table I. These data represellt two samples (freannents 1 and 2) of bivariate data (y, and Y J with sample size n, "" 4 and n, = 3, respectively. Table 1. Sample data to illustrate program. Treatment 1 Y, !dIJ f wid"~ =2 for squared be the average distance I<J cards; 4 I 34 where Wi is an arbitrary weighting !if > 0 with i=1f W; = I Y, Y, 6 3 3 4 7 4 2 7 6 3 2 3 Data MRPP; Input Y, Y,; i=1 constant chosen to be w, = Treatment 2 A SAS data set was created in the usual way and used as input into the IML program. Alternatively, the data can be input directly into the Y matrix in IML. The following SAS statements provide an example of how to use the program. Expressions in bold face may need to be modified for the particular application at hand. between subjects for all subjects within the ;"' group, (; = 1,2, ... , g). Then the t~ statistic of interest is the weighted average ofthe d; values and is given by d= Y, 4 [~I (YkI-Ykl)2Y where v = I for Euclidean distance, v Euclidean distance, etc. Let Program Execution in this paper. 1028 Example: Application of an IN VIVO Receptor Binding Assay 6 3 PROCIML; Start; UseMRPP; Read ALL into Y ; j* Begin User-Specified Parameter Settings *j N_Vec = { 4, 3 } ; j* Sample Size Vector *j M = I( NCOL(Y) ); j* Transfonnation Matrix *j j* End User-Specified Parameter Settings *j Mercante and Johnson [8) presented data to illustrate use of the multivariate permutation test. Data on thirteen mice were collected to assess the acute effects of lorazepam and alprazolam on the. binding of 3!lC i [3H]RoI5-1788 in selected brain regions. The mice were randomly assigned to 3.2 mglkg lorazepam, 1.8 mglkg alprazolam ot saline. AU mice underwent chronic administration of the treatment and received 3 !lCi [3 H]R. 15 - 1788 approximately 24 hours following the chronic phase of treatment. After twenty minutes the mice were decapitated and their brain tissue analyzed. Binding in cpmlmg was determined in six sections of .the brain: Y I = cerebellum, Y, = brain stem,Y, = cortex, Y. = hypothalamus, Y, = striatum and Y. = hippocampus. (remainder ofIML program). .The permutation test statistics for the 35 permutations comprise the complete null distribution Permutation number one, shown in Table 2. corresponding to the original data (ie, no permutation of the data) was found to be the most extreme over the pennutation distribution (p-value = 1/35). We conclude that there is a statistically significant difference in the response distribution for the two treatments. The responses Y I tended to be higher for Treatment 2 but the converse trend was observed for response Y,. For the observed data the MRPP test statistic was - d = 1l.3. From a total of 90,090 permutations, 3,585 yielded more extreme test statistic values giving a p-value of 0.0398. For purposes of comparison, the approximate test resulted in a A traditional multivariate p-value of 0.0354. analysis of variance yielded the following: Test Table 2. All possible test statistics for the multiple response permutation test performed on the data in Table I. Perm Perm # Test Stat Perm # Test Stat I 2.79 13 4.06 25 2 4.3 2.98 14 4.07 26 4.32 3 3.29 15 4.09 27 4.33 4 3.59 16 4.15 28 4.33 5 3.6 17 4.15 29 4.34 # Test Stat 6 3.65 18 4.1 7 30 4.34 l ,l 7 3.72 19 4.18 31 4.4 i 8 3.81 20 4.19 32 4.42 9 3.84 21 4.2 33 4.43 10 3.85 22 4.22 34 4.46 II 3.94 23 4.24 35 4.52 12 4.06 24 4.28 i ,~ P-Value Wilk'sLR 0.094 Pillai's Trace 0.089 Hotelling-Lawley Trace 0.119 Roy's Greatest Root 0.025 Thus, the MRPP detected a statistically significant difference among the groups and in this example appears to be more sensitive to group differences compared to the traditional multivariate parametric tests except for Roy's Greatest Root. Example: Application in Generalized Anxiety Disorder A double-blind, placebo-controlled randomized clinical trial was conducted to investigate the possible axiolytic properties of a psychotropic compound. Nine subjects were randomly allocated to the placebo group and fifteen to the active treatment group. Five response variates were observed: anxiety as determined using the total score from the 21-item Hamilton Anxiety Scale 1029 Severity of illness and depression as measured on the SCL-90 instrument were not statistically significant. The M matrix used to generate a transformation of the data suitable for testing no treatment by time interaction (or, equivalently, parallel treatment profiles over time) is given by M = M'®Is where (Ham A), depression as determined using the total score from the Hamilton Depression Scale (Ham D), severity of illness (SJ), depression as determined using the total score from the depression cluster of the SCL-90 questionnaire, and anxiety as determined using the total score from the anxiety cluster of the SCL-90 questionnaire. The patients were required to take the medication daily throughout the 10-week study. They were also required to return to the clinic biweekly for evaluation resulting in response values measured on each of 6 visits. Each response vector can thus be viewed as multivariate longitudinal data and are described in further detail in Mercante and Johnson [9]. There were a total of 5 x 6 = 30 possible responses for each patient. A few patients dropped out of the study and so data were missing on their later visits. We used the last observation carried forward method to substitute data for the missing values. -I 0 0 0 0 o -I 0 0 0 o 0 -I 0 0 o 0 0 -I 0 o 0 0 0 -I Because of the small sample sizes, a Hotelling T 2 test for treatment by time interaction on all 5 types,of response simultaneously, was not feasible. However, the permutation test was statistically significant (p=0.023). We can average the response profiles over time by iaking M = 16®Is where 16 is a 6xl vector of ones. We should be cautious in' our interpretation of the second test if the corresponding test for interaction is significant. The results are presented here merely to illustrate the test constructions. The total number of subjects in the study was 24 with 30 response variables and therefore an overall multivariate analysis of variance using traditional parametric methods was not possible. The permutation tests were carried out on the transformed data obtained by multiplying the 2x30 data matrix by the appropriate M matrix (chosen to generate functions of the daia relevant to the analysis). The test of the global hypothesis of no between treatment group location shift in the Ix30 response vectors was obtained by iaking the transformation matrix M to be the 30x30 identity matrix I,.. The p-values were 0.037 and 0.035 for the exact and approximate tests, respectively. Sub-hypotheses pertaining to the five scale types were also tested by iaking M to select the response at each of the six time points for each variate in turn. For example, for the sub-hypothesis pertaining to Ham A, we take Hotelling's T2 tests for treatment effect on averages over time were not significant. Conversely, the permutation test was significant for all variate types except SCL-90 depression. Here we see an entirely different interpretation of these data because the power of the parametric multivariate test is so low in this case. Even when response is averaged over time or linear effects over time are considered, the conclusions that can be made are in conflict with those resulting from the permutation tests. Concluding Remarks The fact that Hotelling's T2 and other traditional multivariate tests lose power as the number of response variables increases relative to the sample size is well known. Multivariate permutation tests offer a method of analysis that is not only theoretically sound and requires minimal assumptions for validity but is also valid when the number of response variates exceeds the total sample size. Software for the multiple response permutation tests was available previously in the SAS Supplemental Library as PROC MRPP but has been discontinued in SAS Version 6. We have developed the IML program in SAS Version 6 to fill this void. We M-[ 0::.6 ] where 0""" is a 24x6 matrix of zeros. The tests corresponding to the five sub-hypotheses yielded statistically sigDificant tests for Hamilton anxiety and depression. The tests for severity of illness and SCL-90 depression were borderline significant while that for SCL-90 anxiety was not statistically significant. From these results, we conclude that the active treatment was effective in decreasing both anxiety and depression as measured on the Hamilton scales. 1030 illustrated its use in three specific examples. However, it can be used in a wide range of problems in data analysis. The method is appropriate when the sample size is small relative to the number of response variates and is especially useful for perfonning global testing when the number of response variates exceeds the sample size. The response vectors must be measured on at least the ordinal scale, otherwise, many of the same strategies employed in parametric data analysis can be used. This may require linear transformations on the raw data as in the third example. 3. Mielke PW, KJ Berry and GW Brier, Applications of multi-response permutation procedures for examining seasonal changes in monthly mean sea-level pressure patterns, Monthly Weather Review 109 (1981) 120126. 4. Mielke PW, KJ Berry, PJ Brockwell, and JS Williams, A class of nonparametric tests based on multi-response permutation procedures, Biometrika 68 (1981) 720-724. 5. Berry KJ and PW Mielke, Computation of finite population parameters and approximate probability values for multi-response permutation procedures (MRPP), Communications in Statistics: Simulation and Computations B-12 (1983) 83-107. 6. Berry KJ and PW Mielke, Computation of exact probability values for multi-response permutation procedures (MRPP), Communications in Statistics: Simulation and Computations 13-3 (1984) 417-432. 7. Zimmerman GM, H Goetz and PW Mielke, Use of an improved statistical method for group comparisons to study the effects of prairie fire, Ecology 66-2 (1985) 606-611. 8. Mercante DE and WD Johnson, The analysis of small sample multivariate data with· applications in cliuical trials, Journal of Biopbarmaceutical Statistics, in press. 9. Mercante DE and WD Johnson, The analysis of small sample multivariate data in clinical trials, Proceedings of the Biopharmaceutical Subsection, Joint Meetings of the American Statistical Association, Biometric Society and IMS, Boston, 1992. SAS is a registered trademark of SAS Institute, Inc. In the USA and other countries. ® indicates USA registration. Autbo..s· CUJ"rent Address: Department of Biometry and Genetics Louisiana State University Medical Center 190 I Perdido Street New Orleans, LA 70112-1393 (504) 568-6150 BTINET: PATHDEM@NNOMED R.eferences I. Mielke PW, On asymptotic non-normality of null distributions of MRPP statistics, Communications in Statistics: Theory and Methods A8-15 (1979) 1541-1550. 2. Mielke PW, KJ Berry and ES Johnson, Multi-response permutation procedures for a priori classifications, Communications in Statistics: Theory and Methods AS-14 (1976) 1409-1424. r Ig I t, I l 1031