Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Chapter 8_Field_2005: Comparing several means: ANOVA (General Linear Model, GLM 1) So far we can compare 2 groups/conditions and assess 1 dependent variable E.g., 1 group seeing pictures of and 1 group seeing real spiders (independent variable); measuring their level of anxiety (dependent var) If we want to compare more than 2 groups/ conditions and assess 1 dep var, we have to use an Analysis of Variance (ANOVA) Chapter 8 is about independent ANOVA, i.e, where the various groups/conditions consist of different subjects The problem: experimentalwise error rate Suppose we had 3 groups – why not carry out 3 independent t-tests between those 3? because the probability p of making Type 1 error rises dramatically Remember: for 1 test, the p of not making any Type 1 error is = .95. For 3 tests, this p is .95 x .95 x .95 = .857 Hence, the p of committing a Type 1 error is 14.3%! Even worse for more groups! G1 G1 -G3 G3 G1 -G2 G2 G2 -G3 The solution: ANOVA ANOVA gives us an overall F-value that tells us whether n means are the same, in an overall test (omnibus test). It looks for an overall experimental effect. It does NOT tell us which of the group differences cause this overall effect, only that X1 = X2 = X3 is NOT true. Separate contrasts may be carried out later ANOVA as regresssion Correlational research Looked at real-world relationships Had adopted multiple regression Experimental research Conducted controlled experiments Had adopted ANOVA Although having arisen in different research contexts, regression and ANOVA are conceptually the same thing! The General Linear Model (GLM) accommodates this fact. ANOVA as regression ANOVA compares the amount of systematic variance to the unsystematic variance. This ratio is called F-ratio: ANOVA: F-ratio: systematic (experimental) variance unsystematic (other and error) variance Regression: F-ratio: variance of the regression model variance of the simple model ANOVA as regression The F-ratios in ANOVA and regression are the same. ANOVA can be represented by the multiple regression equation with as many predictors as the experiment has groups minus 1 (because of the df's) An example (using viagra.sav) We compare 3 independent groups Indep Var (IV): treating subjects with 'Viagra' 1. placebo 2. low dose of V 3. high dose of V Dep Var (DP): measure of 'libido' General equation for predicting libido from treatment Outcomei= (Modeli) + errori In a regression approach, the 'experimental model' and the 'simple model' were coded as dummy variabes (1 and 0). The outcome of the good model was compared to that of the simple model. Data in Viagra.sav Placebo Low dose High dose 3 5 7 2 2 4 1 4 5 1 2 3 4 3 6 Mean 2,2 3,2 5 SD 1,3 1,3 1,58 Variance 1,7 1,7 2,5 Grand mean = 3,467Grand SD = 1,767 Grand variance = 3,124 Dummy variables for coding groups With more than two groups, we have to extend the number of dummy variables in regression. If there are only 2 groups, we have a base category 0 and a real model 1. If there are more than 2 groups, we need as many coding variables as there are groups – 1. One group acts as a 'baseline' or 'control group' (0) In the viagra example, there are 3 groups, hence we need 2 coding variables (for the groups 'lw dose' and 'high dose') + a control group ('placebo') Equation for the Viagra-expl Outcome Variable (8.2) Libidoi = b0 + b2Highi + b1Lowi + i Intercept = constant where the regression line cuts the y-axis =mean of placebo group Regression coefficient for the high group the comparison between the 2nd group and the placebo group Regression coefficient for the low group: comparison between the 1st group and the placebo group Error A person'si libido can be predicted from knowing his group membership + the intercept b0. Note that we do not choose the overall mean (grand mean) as a base model (b0) but the mean of the control group, the placebo group. Choice of b0 – the constant So far, in linear regression we had chosen the grand mean as the base model. In binary logistic regression we had chosen the most frequent case as the base model. Here, we choose the placebo group as our base model (b0). Why, do you think? This is because as good experimenters we have to control for any unspecific effects due to giving subjects any treatment. This we test with the placebo group. The comparison between the placebo and the experimental groups will tell us whether, above this unspecific effect, the drug (in various doses) has a causal effect on the dep var. Note that in regression you are free to choose your base model! That makes regression a highly flexible method. Dummy coding of the 3 groups Dummy var 1 Dummy Var 2 Group (High y) (Low y) Placebo 0 0 Low Viagra 0 1 High Viagra 1 0 Each group is now individuated by its particular code Why can't we call the 3 groups: 0,1, 2, or 1,2,3? Because in regression we use these dummy variables in the regression equation when we set the groups to 0 or 1 Equation for the Placebo group Libidoi = b0 + (b2 x 0) + (b1x 0)i Libidoi = b0 XPlacebo = b0 Dummy var 1 Dummy Var 2 Group (High y) (Low y) Placebo 0 0 Low Viagra 0 1 High Viagra 1 0 The predicted value for the placebo group is its mean. This is the intercept/ constant (b0) against which we will test the two experimental groups. Equation for the High-dose group Libidoi = b0 + (b2 x 1) + (b1x 0)i Libidoi = b0 + b2 Dummy var 1 Dummy Var 2 Group (High y) (Low y) Placebo 0 0 Low Viagra 0 1 High Viagra 1 0 XHigh = XPlacebo + b2 b2 = XHigh - XPlacebo The predicted value for the high-dose group is its mean. We compare this mean against the mean of the placebo group. b2 represents the difference between the two groups. Equation for the low-dose group Libidoi = b0 + (b2 x 0) + (b1x 1)i Libidoi = b0 + b1 Dummy var 1 Dummy Var 2 Group (High y) (Low y) Placebo 0 0 Low Viagra 0 1 High Viagra 1 0 XLow = XPlacebo + b1 b1 = XLow - XPlacebo The predicted value for the low-dose group is its mean. We compare this mean against the mean of the placebo group. b1 represents the difference between the two groups. Multiple regression (using dummy.sav) In dummy.sav the three groups (in the column 'dose') are coded with two binary dummy variables (in the columns 'dummy 1' and 'dummy 2'), suited for multiple regression Multiple regression (using dummy.sav) In dummy.sav, the Viagra data of the 3 groups are coded with dummy variables (0/1) as predictors Coefficients The constant/intercept is the mean of the baseline group 'Placebo' For the high-dose group, b2 is 5 – 2.2 = 2.8 For the low-dose group, b1 is 3.2 – 2.2 = 1 b0 is already significant! In a regression analysis the t-tests for the betacoefficients are equal to group comparisons b2 is significant: The difference between the 2 groups (placebo vs. High-dose) is sign. b1 is n.s: The difference between the 2 groups (placebo vs. low-dose) is n.s. Coding dummy-variables for arbitrary group contrasts For 4 groups, you need n-1 (4-1) = 3 dummy variables (DV's). The baseline group is coded 0 throughout. Group 1 Grup 2 Group3 Group 4 (base) DV 1 DV 2 DV 3 1 0 0 0 1 0 0 0 1 0 0 0 The F-ratio In regression, the F-ratio tests the overall fit of a regression model to a set of observed data The same logic applies in ANOVA: we want to test if the means of n groups – in the Viagra example, 3 groups – are different. Thus, the Nullhypothesis is: all means are equal (i.e., they all equal the 'grand mean'.) ANOVAb Sum of Model Squares df Mean Square F Sig. 1 Regression 20,133 2 10,067 5,119 ,025a Residual 23,600 12 1,967 Total 43,733 14 a. Predictors: (Constant), DUMMY2 Dummy Variable 2, DUMMY1 Dummy Variable 1 b. Dependent Variable: LIBIDO Libido Output from Multiple regression with dummy.sav The Viagra data, graphically L i b i d o b2* b1 n.s. Subject (n=15) B1: difference between 'placebo – low dose', n.s. B2: difference between 'placebo – high dose', * The Null-hypothesis is rejected: there are differences between the groups (due to the significant b2) The logic of ANOVA The grand mean represents the case when all groups are equal (H0). If the groups do not differ, the data will all cluster around the grand mean If the b-values of the groups do differ, the regression line will differ from the grand mean. This regression line will represent the data better (H1) The regression model will explain some variance and leave some variance unexplained: F ratio: systematic (experimental) variance unsystematic (other and error) variance The amount of explained variance is bigger than the amount of unexplained variance Basic equation (8.3) Deviation = (observed – model)2 With equation (8.3) we calculate the fit of the basic model (placebo group, b0) and the fit of the best model (regression model, b1 and b2). If the regression model is better than the basic model, it should fit the data better, hence the deviation between the observed data and the predicted data will be smaller. Total sum of squares (SST) For all data points, we sum up their differences from the grand mean and square them. This is SST. (8.4) SST = (xi – xgrand )2 The grand variance is the SST/(N-1). From the grand variance we can caculate the SST. Calculating SST from the grand variance ANOVAb SST = s2 grand (n-1) = 3.124 (15-1) Sum of Model Squares df Mean Square F Sig. 1 Regression 20,133 2 10,067 5,119 ,025a Residual 23,600 12 1,967 Total 43,733 14 a. Predictors: (Constant), DUMMY2 Dummy Variable 2, DUMMY1 Dummy Vari b. Dependent Variable: LIBIDO Libido = 3.125 x 14 = 43.74 Descriptive Statistics Std. N Sum Mean Variance Statistic Statistic Statistic Std. Error Deviation Statistic Statistic LIBIDO Libido 15 52,00 3,4667 ,4563 1,7674 3,124 Valid N (listwise) 15 Grand Variance Degrees of freedom, df For any number of observations, the degrees of freedom is (n-1) since you can only choose (n-1) values freely, whereas the last obervation is determined – it is the only one that is left. If we have 4 observations, they are free to vary in any way, i.e., they can take any value. If we use this sample of 4 observations to calculate the SD of the population (i.e., SE), we take the mean of the sample as an estimate of the population's mean. In doing so, we hold 1 parameter (the mean) constant. Degrees of freedom, df – continued If the sample mean is 10, we also assume that the population mean is 10. This value we keep constant. With the mean fixed, our observations are not free to vary anymore, completely. Only 3 can vary. The fourth must have the value that is left over in order to keep the mean constant Expl: If 3 observations are 7,15, and 8, then the 4th must be 10, in order to arrive at a mean of 10. Thus, holding one parameter constant, the df are one less than the N of the sample, df = n-1 Therefore, if we estimate the SD of the population (SE) from the SD of the sample, we divide the SS by n-1 and not by n. Model Sum of Squares SSM The overall variance is SST = 43.74 Now we want to know how much of this variance is 'model' variance SSM, i.e., can be explained by the variation between the three groups. For each participant, the predicted value is the mean of its condition, i.e., 2.2 for the placebo group, 3.2 for low dose and 5 for high dose. The difference of these predicted values from the grand mean are calculated, squared, multiplied by the n in each group and summed up in SSM. 5 3.2 2.2 Grand mean N of subjects in each group Calculating SSM SSM = nk (xk – Placebo group 2 xgrand) Low dose group High dose group SSM = 5(2.2 – 3.467)2 + 5(3.2 – 3.467)2 + 5(5 – 3.467)2 = 5(-1.267)2 + 5(-0.267)2 + 5(1.533)2 ANOVAb = 8.025 + 0.355 + 11.755 = 20.135 Sum of Model Squares df Mean Square F 1 Regression 20,133 2 10,067 5,119 Residual 23,600 12 1,967 Total 43,733 14 a. Predictors: (Constant), DUMMY2 Dummy Variable 2, DUMMY1 D b. Dependent Variable: LIBIDO Libido The df of SSM is the number of parameters k (3 groups) minus 1 = 3-1=2 Residual Sum of Squares SSR We know already: SST = 43.74 and SSM = 20.14 SS R = SST - SSM SS = 23.6 R ANOVAb Sum of Model Squares df Mean Square F Sig. 1 Regression 20,133 2 10,067 5,119 ,025a Residual 23,600 12 1,967 Total 43,733 14 a. Predictors: (Constant), DUMMY2 Dummy Variable 2, DUMMY1 Dummy Varia b. Dependent Variable: LIBIDO Libido For nerds: calculating the Residual Sum of Squares SSR properly SSR = xik - xk)2 For nerds: calculating the Residual Sum of Squares SSR properly SSR = ( xik– xk)2 This equation is identical to: SSR = SSgroup1 + SSgroup2 + SSgroup2 Variance for each group k SSR = s2k (nk - 1) Number of participants in each group For nerds: calculating the Residual Sum of Squares SSR properly ANOVAb Sum of Model Squares df Mean Square F 1 Regression 20,133 2 10,067 5,119 Residual 23,600 12 1,967 Total 43,733 14 a. Predictors: (Constant), DUMMY2 Dummy Variable 2, DUMMY1 b. Dependent Variable: LIBIDO Libido Placebo Low dose High dose 3 5 7 Sig. ,025 2 2 4 1 4 5 Dummy Variable 1 1 2 3 4 3 6 Mean 2,2 3,2 5 SD 1,3 1,3 1,58 Variance 1,7 1,7 2,5 Grand mean = 3,467Grand SD = 1,767 Grand variance = 3,124 a SSR = s2k (nk - 1) SSR= (1.7) (5-1) + (1.7) (5-1) + (2.5) (5-1) = 6.8 + 6.8 + 10 DfR = dfT – dfM = 23.60 = 14 – 2 =12 Average amount of Variation explained by the model systematic variation Mean squares MS (average SS) MS is the SS divided by the df MSM = SSM = 20.135 = 10.067 dfM 2 MSR = SSR = 23.60 = 1.967 Average amount of Variation not dfR 12 explained by ANOVAb Sum of Model Squares df Mean Square F Sig. 1 Regression 20,133 2 10,067 5,119 ,025a Residual 23,600 12 1,967 Total 43,733 14 a. Predictors: (Constant), DUMMY2 Dummy Variable 2, DUMMY1 Dummy Variable 1 b. Dependent Variable: LIBIDO Libido the model unsystematic variation F-ratio The F-ratio is the ratio between the systematic (explained) and unsystematic (unexplained) variable: F = MSM MSR Commonsense logic: If F < 1, then the t-test must be n.s. since then the unsystematic variance MSR is bigger than the systematic variance MSM. ANOVAb F-ratio for Viagra data F = MSM MSR = 10.067 1.967 Sum of Model Squares df Mean Square F Sig. 1 Regression 20,133 2 10,067 5,119 ,025a Residual 23,600 12 1,967 Total 43,733 14 a. Predictors: (Constant), DUMMY2 Dummy Variable 2, DUMMY1 Dummy Vari b. Dependent Variable: LIBIDO Libido = 5.12 In order to know if 5.12 is significant * we have to test it against against the maximum value we would expect to get by chance alone look up F-distribution, Appendix A.4 ( p 756) with a df=2 (numerator) and df=12 (denominator) , we find a criticial value of 3.89 (p = .05) and 6.93 (p = .01) the F-ratio is * on the 5%-level AssumptionsHow of ANOVA robust is ANOVA? Same assumptions as for any parametric test: Normal distribution Homogeneous variances observations should be independent Interval scale level Homogeneity: Robust when sample sizes are equal Very sensitive to dependent observations Interval scale:Robust even with dichotomous variables, but df should be high (20) Planned contrasts – post hoc tests The F-ratio is a global ratio and tells us whether there is a global effect of the experimental treatment, BUT, if the F-value is significant *, it does NOT tell us which differences (bcoefficients) made the effect significant. It could be that b1 or b2 or both are * Thus, we need to inquire further Planned comparisons or Post hoc tests 1. Planned contrasts In 'planned contrasts' we break down the overall variance into component parts. It is like conducting a 1-tailed test, i.e., when you have a directed hypothesis. Expl. Viagra: 1st contrast: two drug conditions > placebo 2nd contrast: high dose > low dose Partitioning SSM SST (43.73) Total variance in data SSM (20.13) Systematic variance ANOVA SSR (23.6) Unsystematic variance SSM (20.13) Systematic variance Low + high dose Variance explained by experimental groups Low dose High dose Placebo Variance explained by control For comparing 'low dose' vs placebo, we need to carry out a post-hoc test! 1st contrast 2nd contras Rationale of planned contrasts Planned contrasts are like slicing up a cake: once you SSM have cut out one piece, it is unavailable for the next comparison (k-1) contrasts: The number of contrasts is always one less than the number of groups Each contrast compares only 2 chunks of variance The first comparison should always be between all the experimental groups and the control group(s) High dose SSR Low dose Plac ebo SSM Variance explained by experiment Four groups: E1, E2, E3, and C1 Experimental groups E1,2,3 E1 and E2 E1 E2 Control group C1 E3 Planned comparisons in an experiment with 3 Exp and 1 C group Contrast 1 Contrast 2 Contrast 3 SSM Variance explained by experiment Four groups: E1, E2 and C1, C2 Experimental groups E1and E2 E1 Control groups C1and C2 E2 Contrast 1 Contrast 2 Planned comparisons in an experiment with 2 Exp and 2 C- groups C1 C2 Contrast 3 Carrying out contrasts In a planned contrast, we compare 'chunks of variance'. Each chunk can represent more than one group When we carry out contrasts, we assign values to the dummy variables in the regression model With the values we specify which groups we want to compare The resulting coefficients (b1, b2) represent the comparisons The values are called weights We compare any group with a positive weight against any group with a negative weight Rules for carrying out contrasts Rule 1: Choose sensible comparisons. Rule 2: One chunk of variation is coded with a positive value; the other chunk with a negative value. Rule 3: The sum of weights in a contrast is 0. Rule 4: A group that is not included in the contrast is coded as 0. It does not participate in the contrast Rule 5: The weights assigned to the group(s) in one chunk of variation should be equal to the number of groups in the oppositive chunk of variation Weights for the Viagra Expl, 1st contrast SSM (20.13) Systematic variance Low + high dose Placebo Rule 2: Positive sign of weight negative each 1 magnitude of weight 2 Rule 5: +1 +1 weight -2 Rule 3: +1 + +1 weights = 0 Product + -2 = 0 DV 1 DV2 Contrast1 x Group (contrast1) (contrast2) Contrast 2) Placebo -2 0 0 Low dose 1 -1 -1 High dose 1 1 1 Total 0 0 0 Weights for the Viagra Expl, 2nd contrast Chunk 1 Low dose Chunk 2 High dose Rule 2: Positive 1 Rule 5: +1 Rule 3: +1 + Placebo not included negative sign of weight 1 magnitude of weight -1 weight -1 weights =0 Product DV 1 DV2 Contrast1 x Group (contrast1) (contrast2) Contrast 2) Placebo -2 0 0 Low dose 1 -1 -1 High dose 1 1 1 -0 0 0 Regression equation for coding of contrasts (8.9) Libidoi = b0 + b1Contrast1 + b2Contrast2 NOW B 0 represents the grand mean B 1 represents contrast1 (exp groups vs. Placebo) B 2 represents contrast2 (low vs. High dose) Again, you see that you may choose your base model (b0) freely, according to a given model that you want to test Orthogonal contrasts for Viagra Expl Product DV 1 DV2 Contrast1 x Group (contrast1) (contrast2) Contrast 2) Placebo -2 0 0 Low dose 1 -1 -1 High dose 1 1 1 Total 0 0 0 Important: All sums (total),over each variable and over the Product of the contrasts, have to be 0! This means that the contrasts are independent of each other, i.e., orthogonal (in a 90° angle) Independence prevents the error in the t-test to inflate! For nerds: dummy coding and regression The intercept (b0) is equal to the grand mean: b0 = grand mean = (XHigh + XLow + Xplacebo)/3 Regression equation, using the contrast codes of the first contrast Libidoi = b0 + b1Contrast1 + b2Contrast2 Product DV 1 DV2 Contrast1 x Group (contrast1) (contrast2) Contrast 2) Placebo -2 0 0 Low dose 1 -1 -1 High dose 1 1 1 Total 0 0 0 -2 for contrast 1 XPlacebo= XHigh+XLow +XPlacebo + (-2b1) + (b2 x 0) 3 0 for contrast 2 First contrast – continued After rearranging, we get: 2b1 = (XHigh+XLow +XPlacebo )/3 - XPlacebo After multiplying by 3, we get: 6b1 = XHigh+XLow +XPlacebo - 3XPlacebo 6b1 = XHigh+XLow - 2XPlacebo First contrast – continued After dividing by 2, we get 3b1 =(XHigh+XLow )/2 - XPlacebo After dividing by 3, we get b1 = 1/3((XHigh+Xlow/2 ) -Xplacebo) First contrast – continued 3b1 = (XHigh+Xlow)/2 - XPlacebo 3b1 = (5 + 3.2 -2.2)/2 After dividing by 3, we get: b1 = .633 = 1.9 b1 = the regression coefficient that represents the 1st contrast: both exp. groups vs. the control group Note that by dividing by 3 (the number of groups in the contrast), we get a third of the difference between experimental and control groups, hence, the familywise error is controlled for. Regression equation for second contrast Product DV 1 DV2 Contrast1 x Group (contrast1) (contrast2) Contrast 2) Placebo -2 0 0 Low dose 1 -1 -1 High dose 1 1 1 Total 0 0 0 Now we take the dummy coding for the high-dosegroup and insert it into the general equation Libidoi = b0 + b1Contrast1 + b2Contrast2 Xhigh = b0 + (b1 x 1) + (b2 x 1) b2 = Xhigh – b1 – b0 Regression equation for second contrast b2 = Xhigh – b1 – b0 We insert the values for b1 and b0, and get: b1 b0 b2 =Xhigh – (1/3 (XHigh+XLow )/2 – Xplacebo) - (XHigh+XLow +XPlacebo )/3 3b2 = 3Xhigh – ((XHigh+Xlow)/2) – Xplacebo) - (XHigh+XLow +XPlacebo) Second contrast – continued After multiplying by 2, we get: 6b2 = 6XHigh - (XHigh +XLow - 2 XPlacebo) – 2 (XHigh +XLow +XPlacebo) 6b2 = 6XHigh - XHigh +XLow - 2 XPlacebo – 2 XHigh - 2XLow - 2XPlacebo 6b2 = 3XHigh - 3XLow b2 = 1/2 (XHigh - XLow ) b2 = 5 – 3.2 2 = 0.9 By dividing by 2 the familywise error is controlled b2 is the regression coefficient for the 2nd contrast: Low dose vs. High dose Running multiple regression, using 'contrast.sav' 'contrast.sav' is a file in which the Viagra data are coded using the contrast coding scheme. Analyze Regression Linear Outcome: libido predictors: dummy 1, dummy 2 Outcome of multiple regression using 'contrast.sav' Coefficientsa b1 Model 1 (Constant) DUMMY1 Dummy Variable 1 DUMMY2 Dummy Variable 2 Unstandardized Coefficients B 3,467 Std. Error ,362 ,633 ,256 ,900 ,443 a. Dependent Variable: LIBIDO Libido The intercept (b0) is the grand mean Standardi zed Coefficien ts Beta t 9,574 Sig. ,000 ,525 2,474 ,029 Contrast 1 * ,430 2,029 ,065 Contrast 2 n.s. b2 The t-test on the beta weights for the two contrasts show that contrast 1 (both exp. groups vs. control group) is significant contrast 2 (low vs. High dose group) is not significant. Non-orthogonal contrasts You can also carry out contrasts in which dependent comparisons are carried out, e.g. Comparing in your second contrast the high-dose exp group with the placebo group Thus, the first rule “Choose sensible comparisons“ is violated, since you re-use a group (placebo) which you had already singled out in the first contrast The interpretation of such a contrast is problematic A stricter p-level must Product be 1 DV2 Contrast1 x chosen Group DV (contrast1) (contrast2) Contrast 2) Orthogonal contrast Placebo Low dose High dose Total -2 1 1 0 0 -1 1 0 0 -1 1 0 We'd better NOT use non-orthogonal contrasts Product DV 1 DV2 Contrast1 x Group (contrast1) (contrast2) Contrast 2) Placebo -2 -1 2 Low dose 1 0 0 High dose 1 1 1 Total 0 0 3 0 ! Standard contrasts SPSS provides some standard contrast templates, some of which are orthogonal some of which are not. Name Definition Compares the effect of each Deviation Category (except the first) to (first) the overall experimental effect Each category Simple Is compared to (first) The first category Each category ( except Repeated the first ) is compared to The previous category Each category (except the last) Helmert Is compared to the mean effect Of allsubsequent categories The contrasts listet here also have reverse order Contrast 1 2 3 1 2 3 1 2 3 1 2 3 , 3 groups 4 groups 2 vs 1,2,3 2 vs 1,2,3,4 3 vs 1,2,3 3 vs 1,2,3,4 4 vs 1,2,3,4 1 vs 2 1 vs 2 1 vs 3 1 vs 3 1 vs 4 1 vs 2 1 vs 2 2 vs 3 2 vs 3 3 vs 4 1 vs 2,3 1 vs 2,3,4 2 vs 3 2 vs 3,4 3 vs 4 Polynomial contrasts: trend analysis Polynomial contrasts look for trends in the data between ordered groups, if they are linear or not 'Linear' : proportionate increase Quadratic trend: U-shape, 1 inflection point; at least 3 groups of Indep var needed Expl: drug first enhances, then decreases performance Cubic trend: 2 inflection points; at least 4 groups needed Quartic trend: 3 inflection points; at least 5 groups needed (not shown) http://www.gseis.ucla.edu/courses/ed230bc1/cnotes4/trend1.htm l Trend analysis “Inspection of the plot of group means suggests, that although it is possible to fit a straight line through the points, there also seems to be a little bit of curvilinearnity present. Group 1 2 3 4 Linear Quadratic Cubic -3 1 -1 -1 -1 3 1 -1 3 3 1 1 0 0 0 Trend analysis partitions the sum of squares for the model into portions due to linear trend, quadratic trend, cubic trend, etc. If there are k groups it is possible to look at up to k - 1 trends, although often researchers combine together all trends above quadratic or cubic.“ http://www.gseis.ucla.edu/courses/ed230bc1/cnotes4/trend1.htm l Post-hoc procedures When you do not have a prediction about which groups will contrast in what way, you might want to explore any difference between groups that exists. Then you do an exploratory data analysis or data mining. Posthoc tests are a suitable way to do pairwise comparisons between all different combinations of groups. On each of those pairs, a t-test is performed. Keeping the familywise -error small Problem: with each pairwise test, the familywise error rate increases. Solution: dividing by the number of comparisons. Example: If we do 10 comparisons and want to maintain an overall -level of 0.05, we have to conduct single comparisons at the 0.005 level. This procedure is called Bonferroni correction. Problem: we loose statistical power by raising the error (erroneously rejecting our exp. hypothesis) Post-hoc procedures and Type I and II error Least-significant difference (LSD) pairwise comparison does not control type I error at all. Only the overall ANOVA has to be significant Bonferroni's and Tukey's test are the most conservative ones. They keep type I error small but loose statistical power (type II error increases). Various other procedures, among them Welsch Q procedure /REGWQ) which controls type 1 error while having good statistical power. Post-hoc procedures and violations of assumptions What if between the groups the assumptions are not met (normality, equality of variance)? Most multiple comparison procedures tolerate small deviations from normality They do not tolerate unequal group sizes and different population variances SPSS has 4 tests for dealing with different population variances: Tamhane's T2, Dunnett's T3, Games-Howell, Dunnet's C Guidelines for post hoc comparisons When the sample sizes are equal and population variances similar: REGWQ or Tukey Take Bonferroni if you want to cotrol Type I error tightly When sample sizes differ: Gabriel's procedure When equality of variances is doubtful: GamesHowell Running a one-way ANOVA on SPSS (using viagra.sav) The data are organized in columns: 'dose' codes the independent variable (1=placebo; 2=low dose; 3=high dose) libido is the dependent variable This coding looks different from the dummy coding scheme in the regression analysis. However, the dummy codes will be used in the contrasts! The one-way ANOVA tests whether there is a difference between the 3 group means of the dependent variable (libido), as manipulated by your independent variable (dose) Analyze --> Compare Means --> One-way ANOVA (using viagra.sav) Dependent Variable Factor = Independent Variable Dependent Variable: Libido Caution: In an ANOVA you have only 1 dependent variable (SPSS suggests various ones. For analysing > 1 dependent variables you have to use a Multiple ANOVA (MANOVA) Independent Variable ('Factor'): Dose Planned comparisons in SPSS 1. Contrasts: In the dialog box “Contrasts“ specify the kind of contrast you want. We want to check if there is a polynomial trend across the 3 groups and thus tick 'Polynomial' and specify 'Quadratic' (which is the highest trend there can be for 3 groups). 2. Planned comparisons: Specifying the contrasts 1st contrast placebo vs. Low and high dose 2nd contrast Low vs high dose Next Product DV 1 DV2 Contrast1 x Group (contrast1) (contrast2) Contrast 2) Placebo -2 0 0 Low dose 1 -1 -1 High dose 1 1 1 Total 0 0 0 Post-hoc tests in SPSS Bug in the book: Specify “First” Since we have equally sized groups For checking the REGWQ and Tukey We expect: 2 exp groups should be > control group Note: Normally, either you do contrasts OR posthoc tests, NOT both in the same analysis. Options Levene's test checks the assumption of homogeneous variances between groups Produces a graph Note: The Options box in SPSS-10 (above) misses 'Fixed Random effects', 'Brown-Forsythe' and 'Welch' which are present in SPSS-13. 'Brown-Forsythe' and 'Welch' are alternative F-ratios when the assumption of homogeneous variances is violated. Output from one-way ANOVA Descriptives: Gives you a summary of all important descriptive values Descriptives LIBIDO Libido N 1,00 Placebo 2,00 Low Dose 3,00 High Dose Total 5 5 5 15 Mean 2,2000 3,2000 5,0000 3,4667 Std. Deviation 1,3038 1,3038 1,5811 1,7674 Std. Error ,5831 ,5831 ,7071 ,4563 95% Confidence Interval for Mean Lower Bound Upper Bound ,5811 3,8189 1,5811 4,8189 3,0368 6,9632 2,4879 4,4454 Minimum 1,00 2,00 3,00 1,00 Maximum 4,00 5,00 7,00 7,00 Error bar chart Graphs Error bar, simple Define: Choose either SD or SE Error bars represent SD Error bars represent SE: no overlap between placebo and high dose * difference Homogeneity of variances Test of Homogeneity of Variances LIBIDO Libido Levene Statistic df1 ,092 2 df2 12 Sig. ,913 Levene's test is n.s., i.e., the three group do not have differerent variances ANOVA summary table Effects due to the model Can the data be accomodated by a linear model? Experimental effect Mean Squares of the Model MSM ANOVA The linear Model (as well as the linear Model + the quadratic trend) can explain the data sufficiently LIBIDO Libido Between Groups (Combined) Linear Term Contrast Deviation Quadratic Term Contrast Within Groups Total Unsystematic variance Sum of Squares 20,133 19,600 2 1 Mean Square 10,067 19,600 F 5,119 9,966 Sig. ,025 ,008 ,533 1 ,533 ,271 ,612 ,533 23,600 43,733 1 12 14 ,533 1,967 ,271 ,612 Total Sum of Squares SST df Residual Sum of Squares SSR Trend analysis: Is there a quadratic trend in the data? ** ns MSTrend No quadratic trend Mean Squares of the Residual MSR Result There is a significant effect of the experimental conditions (placebo, low, high dose) By looking at the error bars representing SD's, we would not have been able to tell. However, if we take error bars to represent SE's, there is no overlap between placebo and high dose. This corresponds to the significant overall effect. Aside: linear term + quadratic trend How does the linear term and the quadratic term add in the ANOVA? SS Linear term: SS Quadratic term: SS combined 19.6 0.533 20.133 The quadratic trend analysis seeks for a non-linear trend in the data, e.g., for a 'U'- or 'inverted 'U'-shaped curve (see pictures). However, there is none (SSQ-Trend = 0.533). The linear SSM (19.6) is much bigger, hence a linear Model fits the data. The linear term alone can explain more variance (MSM = 19.6) than the combined term (MSCombined = 10.067), however, both together are still significant. Output for 'planned comparisons' 1st contrast: placebo (-2) vs. both exp. groups (Low dose (1) and high dose (1)) 2nd contrast: low dose (-1) against high dose (1) Under an undirected hypothesis (two-tailed) only the 1st contrast is * (0.029) Under a directed hypothesis (1-tailed): The 1st contrast is * The 2nd contrast is * (0.029/2=0.0145; 0.065/2=0.0325) Contrast Coefficients Contrast 1 2 DOSE Dose of Viagra 2,00 Low 3,00 High 1,00 Placebo Dose Dose -2 1 1 0 -1 1 Weighted sum of group means: group mean x weight of the contrast Contrast Tests LIBIDO Libido Assume equal variances Does not assume equal variances Look here in case of unequal variances Contrast 1 2 1 2 Value of Contrast 3,8000 1,8000 3,8000 Std. Error 1,5362 ,8869 1,4832 t 2,474 2,029 2,562 df 12 12 8,740 Sig. (2-tailed) ,029 ,065 ,031 1,8000 ,9165 1,964 7,720 ,086 For nerds: value of contrasts For the values of contrast, you sum up the weighted means of the groups which figure in the contrast: 1st contrast: (XW) = [(2.2 x -2) + (3.2 x 1) + (5 x 1)] = 3.8 2nd contrast: (XW) = [(3.2 x -1) + (5 x 1)] = 1.8 Summary ANOVA 1st contrast: There is a general effect of the experimental treatment: taking viagra increases the libido 2nd contrast: There is a special effect of the two experimental groups: Taking a high dose of viagra increases the libido more than taking a low dose Output for post hoc tests Each group is compaired with each other. We had requested 3 tests: Tukey, Games-Howell, and Dunnett. Multiple Comparisons Dependent Variable: LIBIDO Libido Tukey HSD (I) DOSE Dose of Viagra (J) DOSE Dose of Viagra 1,00 Placebo 2,00 Low Dose 2,00 Low Dose Games-Howell 3,00 High Dose 1,00 Placebo 3,00 High Dose Mean Difference (I-J) Std. Error -1,0000 ,8869 -2,8000* ,8869 1,0000 ,8869 -1,8000 ,8869 Sig. ns ,516 * ,021 ns ,516 ,147 95% Confidence Interval Lower Bound Upper Bound -3,3663 1,3663 -5,1663 -,4337 -1,3663 3,3663 -4,1663 ,5663 3,00 High Dose 1,00 Placebo 2,00 Low Dose 2,8000* 1,8000 ,8869 ,8869 ,021 ,147 ,4337 -,5663 5,1663 4,1663 1,00 Placebo 2,00 Low Dose 3,00 High Dose -1,0000 -2,8000* ,8869 ,8869 -3,3563 -5,4390 1,3563 -,1610 2,00 Low Dose 1,00 Placebo 3,00 High Dose 1,0000 -1,8000 ,8869 ,8869 ns ,479 * ,039 ,479 ns ,185 -1,3563 -4,4390 3,3563 ,8390 3,00 High Dose 1,00 Placebo 2,00 Low Dose 2,8000* 1,8000 ,8869 ,8869 ,039 ,185 ,1610 -,8390 5,4390 4,4390 1,00 Placebo 1,00 Placebo 1,0000 2,8000* ,8869 ,8869 ns ,227 * ,008 Dunnett t (<control)a 2,00 Low Dose 3,00 High Dose Note: in our planned comparisons this contrast was *! 2,8698 4,6698 *. The mean difference is significant at the .05 level. a. Dunnett t-tests treat one group as a control, and compare all other groups against it. Why are some contrasts ns for post hoc tests but * for planned comparisons? Post hoc tests are always 2-sided (you do not have any specific hypothesis); except Dunnett test where we did a 1-tailed test Post hoc tests are always conservative in that they try to avoid inflation of familywise error Homogeneous subsets LIBIDO Libido Tukey HSDa Ryan-Einot-Gabri el-Welsch Range DOSE Dose of Viagra 1,00 Placebo 2,00 Low Dose 3,00 High Dose Sig. 1,00 Placebo 2,00 Low Dose 3,00 High Dose Sig. N 5 5 5 5 5 5 Subset for alpha = .05 1 2 2,2000 3,2000 3,2000 5,0000 ,516 ,147 2,2000 3,2000 3,2000 5,0000 ,282 ,065 Means for groups in homogeneous subsets are displayed. a. Uses Harmonic Mean Sample Size = 5,000. SPSS creates subsets of groups with similar means, here placebo + low dose (1) and low dose + high dose (2). Within those groups, there is ns difference Calculating the effect size (overall ANOVA) Since SPSS does not routinely caculate the effect size, we have to do it ourselves. Since we know R2, we can derive r (effect size) from it: R2 = SSM = 20.13 = .46 SST 43.73 r = .46 = .68 Since .68 is > .5, the benchmark of a big effect, we can safely claim that our finding is a substantial one. An alternative and more exact measure is which estimates the effect size in the population. (= .60). Calculating the effect size (single contrasts) rcontrasts = t rcontrast 1 = 2.474 / 2.4742 + 12 = .58 big effect rcontrast 2 = 2.029 / 2.0202 + 12 = .51 big effect 2 /t2 + df 2 2 Reporting 1-way ANOVA ( Field, 2005, p 359) There was a significant effect of Viagra on levels of libido, F (2,12) = 5.12, p < .05, = .60. There was a significant linear trend, F (1,12) = 9.97, p < .01, = .62, indicating that as the dose of Viagra increased, libido increased proportionately. Planned contrasts revealed that having any dose of Viagra significantly increased libido compared to having a placebo, t (12) = 2.47, p < .05 (onetailed), r = .58, and that having a high dose significantly increased libido compared to having a low dose, t (12) = 2.03, p < .05 (one-tailed), r = .51.