Download Chapter 8_Field_2005: Comparing several means: ANOVA

Chapter 8_Field_2005:
Comparing several means: ANOVA
(General Linear Model, GLM 1)
So far we can compare 2 groups/conditions and
assess 1 dependent variable
E.g., 1 group seeing pictures of and 1 group
seeing real spiders (independent variable);
measuring their level of anxiety (dependent var)
If we want to compare more than 2 groups/
conditions and assess 1 dep var, we have to use
an Analysis of Variance (ANOVA)
Chapter 8 is about independent ANOVA, i.e,
where the various groups/conditions consist
of different subjects

The problem: experimentalwise error rate
Suppose we had 3 groups –
why not carry out 3
independent t-tests between
those 3?
 because the probability p of
making Type 1 error rises
dramatically
Remember: for 1 test, the p of
not making any Type 1 error is
= .95.
For 3 tests, this p is .95 x .95
x .95 = .857
Hence, the p of committing a
Type 1 error is 14.3%!
Even worse for more groups!

G1
G1 -G3
G3
G1 -G2
G2
G2 -G3
The solution: ANOVA
ANOVA gives us an overall F-value that
tells us whether n means are the same, in
an overall test (omnibus test). It looks for an
overall experimental effect.
It does NOT tell us which of the group
differences cause this overall effect, only
that X1 = X2 = X3 is NOT true.
Separate contrasts may be carried out later

ANOVA as regresssion



Correlational research
Looked at real-world
relationships
Had adopted multiple
regression



Experimental
research
Conducted controlled
experiments
Had adopted ANOVA
Although having arisen in different research
contexts, regression and ANOVA are
conceptually the same thing!
The General Linear Model (GLM)
accommodates this fact.
ANOVA as regression
ANOVA compares the amount of systematic
variance to the unsystematic variance. This ratio
is called F-ratio:

ANOVA:
F-ratio: systematic (experimental) variance
unsystematic (other and error) variance
Regression:
F-ratio: variance of the regression model
variance of the simple model
ANOVA as regression
The F-ratios in ANOVA and regression are the
same.
ANOVA can be represented by the multiple
regression equation with as many predictors as
the experiment has groups minus 1 (because of
the df's)
An example (using viagra.sav)
We compare 3 independent groups
Indep Var (IV): treating subjects with 'Viagra'

1. placebo
2. low dose of V
3. high dose of V
Dep Var (DP): measure of 'libido'
General equation for predicting libido from treatment

Outcomei= (Modeli) + errori
In a regression approach, the 'experimental model'
and the 'simple model' were coded as dummy
variabes (1 and 0). The outcome of the good model
was compared to that of the simple model.
Data in Viagra.sav
Placebo
Low dose High dose
3
5
7
2
2
4
1
4
5
1
2
3
4
3
6
Mean
2,2
3,2
5
SD
1,3
1,3
1,58
Variance
1,7
1,7
2,5
Grand mean = 3,467Grand SD = 1,767
Grand variance = 3,124
Dummy variables for coding groups
With more than two groups, we have to extend
the number of dummy variables in regression.
If there are only 2 groups, we have a base
category 0 and a real model 1.
If there are more than 2 groups, we need as
many coding variables as there are groups – 1.
One group acts as a 'baseline' or 'control group'
(0)
In the viagra example, there are 3 groups, hence
we need 2 coding variables (for the groups 'lw
dose' and 'high dose') + a control group ('placebo')

Equation for the Viagra-expl
Outcome Variable
(8.2) Libidoi = b0 + b2Highi + b1Lowi + i
Intercept = constant
where
the regression
line cuts the y-axis
=mean of placebo
group
Regression
coefficient for
the high group
the comparison
between the
2nd group
and the placebo
group
Regression
coefficient for the
low group:
comparison
between the
1st group and
the placebo
group
Error
A person'si libido can be predicted from knowing
his group membership + the intercept b0.
Note that we do not choose the overall mean (grand mean) as a base
model (b0) but the mean of the control group, the placebo group.
Choice of b0 – the constant
So far, in linear regression we had chosen the grand
mean as the base model.
In binary logistic regression we had chosen the most
frequent case as the base model.
Here, we choose the placebo group as our base model
(b0). Why, do you think?
This is because as good experimenters we have to control
for any unspecific effects due to giving subjects any
treatment. This we test with the placebo group. The
comparison between the placebo and the experimental
groups will tell us whether, above this unspecific effect, the
drug (in various doses) has a causal effect on the dep var.
Note that in regression you are free to choose your base
model! That makes regression a highly flexible method.
Dummy coding of the 3 groups
Dummy var 1 Dummy Var 2
Group
(High y)
(Low y)
Placebo
0
0
Low Viagra
0
1
High Viagra
1
0
Each group is now individuated by its
particular code
Why can't we call the 3 groups: 0,1, 2, or
1,2,3?

Because in regression we use these dummy
variables in the regression equation when we
set the groups to 0 or 1
Equation for the Placebo group
Libidoi = b0 + (b2 x 0) + (b1x 0)i
Libidoi = b0
XPlacebo = b0
Dummy var 1 Dummy Var 2
Group
(High y)
(Low y)
Placebo
0
0
Low Viagra
0
1
High Viagra
1
0
The predicted value for the placebo
group is its mean. This is the intercept/
constant (b0) against which we will test
the two experimental groups.
Equation for the High-dose group
Libidoi = b0 + (b2 x 1) + (b1x 0)i
Libidoi = b0 + b2
Dummy var 1 Dummy Var 2
Group
(High y)
(Low y)
Placebo
0
0
Low Viagra
0
1
High Viagra
1
0
XHigh = XPlacebo + b2
b2 = XHigh - XPlacebo
The predicted value for the high-dose
group is its mean. We compare this
mean against the mean of the placebo
group. b2 represents the difference
between the two groups.
Equation for the low-dose group
Libidoi = b0 + (b2 x 0) + (b1x 1)i
Libidoi = b0 + b1
Dummy var 1 Dummy Var 2
Group
(High y)
(Low y)
Placebo
0
0
Low Viagra
0
1
High Viagra
1
0
XLow = XPlacebo + b1
b1 = XLow - XPlacebo
The predicted value for the low-dose
group is its mean. We compare this
mean against the mean of the placebo
group. b1 represents the difference
between the two groups.
Multiple regression (using dummy.sav)
In dummy.sav the three
groups (in the column
'dose') are coded with
two binary dummy
variables (in the columns
'dummy 1' and 'dummy
2'), suited for multiple
regression
Multiple regression (using dummy.sav)
In dummy.sav, the Viagra data of the 3 groups are
coded with dummy variables (0/1) as predictors
Coefficients
The constant/intercept
is the mean of the
baseline group
'Placebo'
For the high-dose
group, b2 is
5 – 2.2 = 2.8
For the low-dose
group, b1 is
3.2 – 2.2 = 1
b0 is already
significant!
In a regression analysis
the t-tests for the betacoefficients are equal to
group comparisons
b2 is significant:
The difference
between the 2
groups (placebo vs.
High-dose) is sign.
b1 is n.s:
The difference
between the 2
groups (placebo vs.
low-dose) is n.s.
Coding dummy-variables for
arbitrary group contrasts
For 4 groups, you need n-1 (4-1) = 3
dummy variables (DV's). The baseline
group is coded 0 throughout.
Group 1
Grup 2
Group3
Group 4 (base)
DV 1 DV 2 DV 3
1
0
0
0
1
0
0
0
1
0
0
0
The F-ratio
In regression, the F-ratio tests the overall fit of a
regression model to a set of observed data
The same logic applies in ANOVA: we want to
test if the means of n groups – in the Viagra
example, 3 groups – are different. Thus, the Nullhypothesis is: all means are equal (i.e., they all
equal the 'grand mean'.)

ANOVAb
Sum of
Model
Squares
df
Mean Square
F
Sig.
1
Regression 20,133
2
10,067
5,119
,025a
Residual
23,600
12
1,967
Total
43,733
14
a. Predictors: (Constant), DUMMY2 Dummy Variable 2, DUMMY1 Dummy Variable 1
b. Dependent Variable: LIBIDO Libido
Output from Multiple regression with
dummy.sav
The Viagra data, graphically

L
i
b
i
d
o
b2*
b1
n.s.
Subject (n=15)


B1: difference
between 'placebo –
low dose', n.s.
B2: difference
between 'placebo –
high dose', *
The Null-hypothesis is
rejected: there are
differences between
the groups (due to the
significant b2)
The logic of ANOVA

The grand mean represents the case when
all groups are equal (H0). If the groups do
not differ, the data will all cluster around
the grand mean
 If the b-values of the groups do differ, the
regression line will differ from the grand
mean. This regression line will represent
the data better (H1)
 The regression model will explain some
variance and leave some variance
unexplained:
F ratio: systematic (experimental) variance
unsystematic (other and error) variance
The amount of explained variance is bigger than
the amount of unexplained variance

Basic equation
(8.3)
Deviation =
(observed – model)2
With equation (8.3) we calculate the fit of the
basic model (placebo group, b0) and the fit of the
best model (regression model, b1 and b2).
If the regression model is better than the basic
model, it should fit the data better, hence the
deviation between the observed data and the
predicted data will be smaller.

Total sum of squares (SST)
For all data points, we sum up their differences
from the grand mean and square them. This is
SST.

(8.4)
SST =
 (xi – xgrand )2
The grand variance is the SST/(N-1).
From the grand variance we can caculate the SST.
Calculating SST from the grand variance
ANOVAb
SST =
s2
grand
(n-1)
= 3.124 (15-1)
Sum of
Model
Squares
df
Mean Square
F
Sig.
1
Regression 20,133
2
10,067
5,119
,025a
Residual
23,600
12
1,967
Total
43,733
14
a. Predictors: (Constant), DUMMY2 Dummy Variable 2, DUMMY1 Dummy Vari
b. Dependent Variable: LIBIDO Libido
= 3.125 x 14
= 43.74
Descriptive Statistics
Std.
N
Sum
Mean
Variance
Statistic Statistic Statistic Std. Error Deviation
Statistic Statistic
LIBIDO Libido
15
52,00 3,4667
,4563 1,7674
3,124
Valid N (listwise)
15
Grand
Variance
Degrees of freedom, df
For any number of observations, the degrees
of freedom is (n-1) since you can only choose
(n-1) values freely, whereas the last
obervation is determined – it is the only one
that is left.
If we have 4 observations, they are free to
vary in any way, i.e., they can take any value.
If we use this sample of 4 observations to
calculate the SD of the population (i.e., SE),
we take the mean of the sample as an
estimate of the population's mean. In doing so,
we hold 1 parameter (the mean) constant.

Degrees of freedom, df – continued
If the sample mean is 10, we also assume that
the population mean is 10. This value we keep
constant.
With the mean fixed, our observations are not
free to vary anymore, completely. Only 3 can vary.
The fourth must have the value that is left over in
order to keep the mean constant
Expl: If 3 observations are 7,15, and 8, then the
4th must be 10, in order to arrive at a mean of 10.
 Thus, holding one parameter constant, the df
are one less than the N of the sample, df = n-1
Therefore, if we estimate the SD of the
population (SE) from the SD of the sample, we
divide the SS by n-1 and not by n.

Model Sum of Squares SSM
The overall variance is SST = 43.74
Now we want to know how much of this variance is
'model' variance SSM, i.e., can be explained by the
variation between the three groups.
For each participant, the predicted value is the
mean of its condition, i.e., 2.2 for the placebo
group, 3.2 for low dose and 5 for high dose.
The difference of these predicted values from the
grand mean are calculated, squared, multiplied by
the n in each group and summed up in SSM.

5
3.2
2.2
Grand mean
N of subjects
in each group
Calculating SSM
SSM = nk (xk –
Placebo group
2
xgrand)
Low dose group
High dose group
SSM = 5(2.2 – 3.467)2 + 5(3.2 – 3.467)2 + 5(5 – 3.467)2
= 5(-1.267)2 + 5(-0.267)2 + 5(1.533)2
ANOVAb
= 8.025 + 0.355 + 11.755
= 20.135
Sum of
Model
Squares
df
Mean Square
F
1
Regression 20,133
2
10,067
5,119
Residual
23,600
12
1,967
Total
43,733
14
a. Predictors: (Constant), DUMMY2 Dummy Variable 2, DUMMY1 D
b. Dependent Variable: LIBIDO Libido
The df of SSM is the number of parameters
k (3 groups) minus 1 = 3-1=2
Residual Sum of Squares SSR
We know already:
SST = 43.74 and SSM = 20.14
SS
R = SST - SSM
SS = 23.6
R

ANOVAb
Sum of
Model
Squares
df
Mean Square
F
Sig.
1
Regression 20,133
2
10,067
5,119
,025a
Residual
23,600
12
1,967
Total
43,733
14
a. Predictors: (Constant), DUMMY2 Dummy Variable 2, DUMMY1 Dummy Varia
b. Dependent Variable: LIBIDO Libido
For nerds: calculating the Residual
Sum of Squares SSR properly
SSR = xik - xk)2




For nerds: calculating the Residual
Sum of Squares SSR properly
SSR = ( xik– xk)2
This equation is identical to:
SSR = SSgroup1 + SSgroup2 + SSgroup2
Variance
for each
group k
SSR = s2k (nk - 1)
Number of
participants
in each group
For nerds: calculating the Residual
Sum of Squares SSR properly
ANOVAb
Sum of
Model
Squares
df
Mean Square
F
1
Regression 20,133
2
10,067
5,119
Residual
23,600
12
1,967
Total
43,733
14
a. Predictors: (Constant), DUMMY2 Dummy Variable 2, DUMMY1
b. Dependent Variable: LIBIDO Libido
Placebo
Low dose High dose
3
5
7
Sig.
,025
2
2
4
1
4
5
Dummy Variable 1
1
2
3
4
3
6
Mean
2,2
3,2
5
SD
1,3
1,3
1,58
Variance
1,7
1,7
2,5
Grand mean = 3,467Grand SD = 1,767
Grand variance = 3,124
a
SSR = s2k (nk - 1)
SSR= (1.7) (5-1) + (1.7) (5-1) + (2.5) (5-1)
= 6.8 + 6.8 + 10
DfR = dfT – dfM
= 23.60
= 14 – 2 =12
Average amount
of Variation
explained by
the model systematic
variation
Mean squares MS (average SS)
MS is the SS divided by the df
MSM = SSM = 20.135 = 10.067
dfM
2
MSR = SSR = 23.60
= 1.967
Average amount
of Variation not
dfR
12
explained by
ANOVAb
Sum of
Model
Squares
df
Mean Square
F
Sig.
1
Regression 20,133
2
10,067
5,119
,025a
Residual
23,600
12
1,967
Total
43,733
14
a. Predictors: (Constant), DUMMY2 Dummy Variable 2, DUMMY1 Dummy Variable 1
b. Dependent Variable: LIBIDO Libido
the model unsystematic
variation
F-ratio
The F-ratio is the ratio between the systematic
(explained) and unsystematic (unexplained)
variable:
F = MSM
MSR
Commonsense logic: If F < 1, then the t-test
must be n.s. since then the unsystematic variance
MSR is bigger than the systematic variance MSM.
ANOVAb
F-ratio for Viagra data
F = MSM
MSR
= 10.067
1.967
Sum of
Model
Squares
df
Mean Square
F
Sig.
1
Regression 20,133
2
10,067
5,119
,025a
Residual
23,600
12
1,967
Total
43,733
14
a. Predictors: (Constant), DUMMY2 Dummy Variable 2, DUMMY1 Dummy Vari
b. Dependent Variable: LIBIDO Libido
= 5.12
In order to know if 5.12 is significant * we have to
test it against against the maximum value we
would expect to get by chance alone
 look up F-distribution, Appendix A.4 ( p 756)
with a df=2 (numerator) and df=12 (denominator) ,
we find a criticial value of 3.89 (p = .05) and 6.93
(p = .01)
 the F-ratio is * on the 5%-level
AssumptionsHow
of ANOVA
robust is ANOVA?
Same assumptions as
for any parametric test:
Normal distribution
Homogeneous
variances
observations should be
independent
Interval scale level



Homogeneity: Robust
when sample sizes are
equal
Very sensitive to
dependent observations
Interval scale:Robust
even with dichotomous
variables, but df should
be high (20)
Planned contrasts – post hoc tests
The F-ratio is a global ratio and tells us whether
there is a global effect of the experimental
treatment,
BUT, if the F-value is significant *,
it does NOT tell us which differences (bcoefficients) made the effect significant. It could
be that b1 or b2 or both are *
Thus, we need to inquire further
Planned
comparisons
or
Post hoc tests
1. Planned contrasts
In 'planned contrasts' we break down the overall
variance into component parts. It is like
conducting a 1-tailed test, i.e., when you have a
directed hypothesis.
Expl. Viagra:
1st contrast: two drug conditions > placebo
2nd contrast: high dose > low dose
Partitioning SSM
SST (43.73)
Total variance in
data
SSM (20.13)
Systematic
variance
ANOVA
SSR (23.6)
Unsystematic
variance
SSM (20.13)
Systematic variance
Low + high dose
Variance explained
by experimental
groups
Low
dose
High
dose
Placebo
Variance
explained
by control
For comparing 'low dose' vs
placebo, we need to carry out a
post-hoc test!
1st contrast
2nd contras
Rationale of planned contrasts
Planned contrasts are like
slicing up a cake: once you
SSM
have cut out one piece, it is
unavailable for the next
comparison
(k-1) contrasts: The number of
contrasts is always one less
than the number of groups
Each contrast compares only 2
chunks of variance

The first comparison should
always be between all the
experimental groups and the
control group(s)

High
dose
SSR
Low
dose
Plac
ebo
SSM
Variance explained by experiment
Four groups: E1, E2, E3, and C1
Experimental
groups
E1,2,3
E1 and
E2
E1
E2
Control
group
C1
E3
Planned comparisons
in an experiment with
3 Exp and 1 C group
Contrast 1
Contrast 2
Contrast 3
SSM
Variance explained by experiment
Four groups: E1, E2 and C1, C2
Experimental
groups
E1and E2
E1
Control
groups
C1and C2
E2
Contrast 1
Contrast 2
Planned comparisons in
an experiment with 2
Exp and 2 C- groups
C1
C2
Contrast 3
Carrying out contrasts
In a planned contrast, we compare 'chunks of
variance'. Each chunk can represent more than
one group
When we carry out contrasts, we assign values to
the dummy variables in the regression model
With the values we specify which groups we want
to compare
The resulting coefficients (b1, b2) represent the
comparisons
The values are called weights
We compare any group with a positive weight
against any group with a negative weight

Rules for carrying out contrasts
Rule 1: Choose sensible comparisons.
Rule 2: One chunk of variation is coded with a
positive value; the other chunk with a negative
value.
Rule 3: The sum of weights in a contrast is 0.
Rule 4: A group that is not included in the
contrast is coded as 0. It does not participate in
the contrast
Rule 5: The weights assigned to the group(s) in
one chunk of variation should be equal to the
number of groups in the oppositive chunk of
variation

Weights for the Viagra Expl, 1st contrast
SSM (20.13)
Systematic variance
Low + high dose
Placebo
Rule 2: Positive
sign of weight
negative
each 1
magnitude of weight 2
Rule 5: +1
+1 weight
-2
Rule 3: +1 +
+1  weights = 0 Product + -2 = 0
DV 1
DV2
Contrast1 x
Group
(contrast1) (contrast2) Contrast 2)
Placebo
-2
0
0
Low dose
1
-1
-1
High dose
1
1
1
Total
0
0
0
Weights for the Viagra Expl, 2nd contrast
Chunk 1
Low dose
Chunk 2
High dose
Rule 2: Positive
1
Rule 5: +1
Rule 3: +1 +
Placebo
not included
negative sign of weight
1
magnitude of weight
-1
weight
-1
 weights
=0
Product
DV 1
DV2
Contrast1 x
Group
(contrast1) (contrast2) Contrast 2)
Placebo
-2
0
0
Low dose
1
-1
-1
High dose
1
1
1
-0
0
0
Regression equation for coding of
contrasts
(8.9) Libidoi = b0 + b1Contrast1 + b2Contrast2
NOW
B
0 represents the grand mean
B
1 represents contrast1 (exp groups vs. Placebo)
B
2 represents contrast2 (low vs. High dose)
Again, you see that you may choose your base
model (b0) freely, according to a given model that
you want to test
Orthogonal contrasts for Viagra Expl
Product
DV 1
DV2
Contrast1 x
Group
(contrast1) (contrast2) Contrast 2)
Placebo
-2
0
0
Low dose
1
-1
-1
High dose
1
1
1
Total
0
0
0
Important:
All sums (total),over each variable and over the
Product of the contrasts, have to be 0!
This means that the contrasts are independent
of each other, i.e., orthogonal (in a 90° angle)
Independence prevents the error in the t-test to
inflate!
For nerds: dummy coding and regression
The intercept (b0) is equal to the grand mean:
b0 = grand mean = (XHigh + XLow + Xplacebo)/3
Regression equation, using the contrast
codes of the first contrast
Libidoi = b0 + b1Contrast1 + b2Contrast2
Product
DV 1
DV2
Contrast1 x
Group
(contrast1) (contrast2) Contrast 2)
Placebo
-2
0
0
Low dose
1
-1
-1
High dose
1
1
1
Total
0
0
0
-2 for contrast 1
XPlacebo= XHigh+XLow +XPlacebo + (-2b1) + (b2 x 0)
3
0 for contrast 2
First contrast – continued
After rearranging, we get:
2b1 = (XHigh+XLow +XPlacebo )/3 - XPlacebo
After multiplying by 3, we get:
6b1 = XHigh+XLow +XPlacebo - 3XPlacebo
6b1 = XHigh+XLow - 2XPlacebo
First contrast – continued
After dividing by 2, we get
3b1 =(XHigh+XLow )/2 - XPlacebo
After dividing by 3, we get
b1
= 1/3((XHigh+Xlow/2 ) -Xplacebo)
First contrast – continued
3b1 = (XHigh+Xlow)/2
- XPlacebo
3b1 = (5 + 3.2 -2.2)/2
After dividing by 3, we get:
b1 = .633
= 1.9
b1 = the regression
coefficient that
represents the
1st contrast:
both exp. groups vs.
the control group
Note that by dividing by 3 (the number of groups
in the contrast), we get a third of the difference
between experimental and control groups, hence,
the familywise error is controlled for.
Regression equation for second contrast
Product
DV 1
DV2
Contrast1 x
Group
(contrast1) (contrast2) Contrast 2)
Placebo
-2
0
0
Low dose
1
-1
-1
High dose
1
1
1
Total
0
0
0
Now we take the dummy coding for the high-dosegroup and insert it into the general equation
Libidoi = b0 + b1Contrast1 + b2Contrast2
Xhigh = b0 + (b1 x 1) + (b2 x 1)
b2
= Xhigh – b1 – b0
Regression equation for second contrast
b2 = Xhigh – b1 – b0
We insert the values for b1 and b0, and get:
b1
b0
b2 =Xhigh – (1/3 (XHigh+XLow )/2 – Xplacebo) - (XHigh+XLow +XPlacebo )/3
3b2 = 3Xhigh – ((XHigh+Xlow)/2) – Xplacebo) - (XHigh+XLow +XPlacebo)
Second contrast – continued
After multiplying by 2, we get:
6b2 = 6XHigh - (XHigh +XLow - 2 XPlacebo) – 2 (XHigh +XLow
+XPlacebo)
6b2 = 6XHigh - XHigh +XLow - 2 XPlacebo – 2 XHigh - 2XLow - 2XPlacebo
6b2 = 3XHigh - 3XLow
b2 = 1/2 (XHigh - XLow )
b2
= 5 – 3.2
2
= 0.9
By dividing by 2
the familywise
error is controlled
b2 is the regression
coefficient for the
2nd contrast:
Low dose vs. High dose
Running multiple regression,
using 'contrast.sav'
'contrast.sav' is a file in which
the Viagra data are coded using
the contrast coding scheme.
Analyze  Regression 
Linear
Outcome: libido
predictors:
dummy 1,
dummy 2
Outcome of multiple regression
using 'contrast.sav'
Coefficientsa
b1
Model
1
(Constant)
DUMMY1 Dummy
Variable 1
DUMMY2 Dummy
Variable 2
Unstandardized
Coefficients
B
3,467
Std. Error
,362
,633
,256
,900
,443
a. Dependent Variable: LIBIDO Libido
The intercept (b0) is
the grand mean
Standardi
zed
Coefficien
ts
Beta
t
9,574
Sig.
,000
,525
2,474
,029
Contrast 1 *
,430
2,029
,065
Contrast 2 n.s.
b2
The t-test on the beta weights for the two contrasts show that
contrast 1 (both exp. groups vs. control group) is significant
contrast 2 (low vs. High dose group) is not significant.

Non-orthogonal contrasts
You can also carry out contrasts in which dependent
comparisons are carried out, e.g. Comparing in your
second contrast the high-dose exp group with the
placebo group
Thus, the first rule “Choose sensible comparisons“ is
violated, since you re-use a group (placebo) which
you had already singled out in the first contrast
The interpretation of such a contrast is problematic

A stricter p-level must Product
be
1
DV2
Contrast1 x
chosen Group DV
(contrast1) (contrast2) Contrast 2)

Orthogonal
contrast
Placebo
Low dose
High dose
Total
-2
1
1
0
0
-1
1
0
0
-1
1
0
 We'd better NOT use
non-orthogonal contrasts
Product
DV 1
DV2
Contrast1 x
Group
(contrast1) (contrast2) Contrast 2)
Placebo
-2
-1
2
Low dose
1
0
0
High dose
1
1
1
Total
0
0
3
0 !
Standard contrasts
SPSS provides some standard contrast
templates, some of which are orthogonal
some of which are not.
Name
Definition
Compares the effect of each
Deviation Category (except the first) to
(first)
the overall experimental effect
Each category
Simple
Is compared to
(first)
The first category
Each category ( except
Repeated the first ) is compared to
The previous category
Each category (except the last)
Helmert Is compared to the mean effect
Of allsubsequent categories
The contrasts listet here also have reverse order
Contrast
1
2
3
1
2
3
1
2
3
1
2
3
,
3 groups 4 groups
2 vs 1,2,3 2 vs 1,2,3,4
3 vs 1,2,3 3 vs 1,2,3,4
4 vs 1,2,3,4
1 vs 2
1 vs 2
1 vs 3
1 vs 3
1 vs 4
1 vs 2
1 vs 2
2 vs 3
2 vs 3
3 vs 4
1 vs 2,3 1 vs 2,3,4
2 vs 3
2 vs 3,4
3 vs 4
Polynomial contrasts: trend analysis
Polynomial contrasts look for trends in the data
between ordered groups, if they are linear or not


'Linear' : proportionate increase
Quadratic trend: U-shape, 1 inflection
point; at least 3 groups of Indep var
needed

Expl: drug first enhances, then decreases performance
Cubic trend: 2 inflection points; at
least 4 groups needed

Quartic trend: 3 inflection points; at
least 5 groups needed (not shown)

http://www.gseis.ucla.edu/courses/ed230bc1/cnotes4/trend1.htm
l
Trend analysis
“Inspection of the plot of group means suggests, that
although it is possible to fit a straight line through the
points, there also seems to be a little bit of
curvilinearnity present. Group
1 2 3 4
Linear
Quadratic
Cubic
-3
1
-1
-1
-1
3
1
-1
3
3
1
1
0
0
0
Trend analysis partitions the sum of squares for the
model into portions due to linear trend, quadratic
trend, cubic trend, etc. If there are k groups it is
possible to look at up to k - 1 trends, although often
researchers combine together all trends above
quadratic or cubic.“
http://www.gseis.ucla.edu/courses/ed230bc1/cnotes4/trend1.htm
l
Post-hoc procedures
When you do not have a prediction about which
groups will contrast in what way, you might want
to explore any difference between groups that
exists. Then you do an exploratory data analysis
or data mining.
Posthoc tests are a suitable way to do pairwise
comparisons between all different combinations of
groups.
On each of those pairs, a t-test is performed.
Keeping the familywise -error small
Problem: with each pairwise test, the familywise
error rate increases.
Solution: dividing  by the number of
comparisons.
Example: If we do 10 comparisons and want to
maintain an overall -level of 0.05, we have to
conduct single comparisons at the 0.005 level.
This procedure is called Bonferroni correction.
Problem: we loose statistical power by raising the
 error (erroneously rejecting our exp. hypothesis)
Post-hoc procedures
and Type I and II error
Least-significant difference (LSD) pairwise
comparison does not control type I error at all.
Only the overall ANOVA has to be significant
Bonferroni's and Tukey's test are the most
conservative ones. They keep type I error small
but loose statistical power (type II error
increases).
Various other procedures, among them Welsch Q
procedure /REGWQ) which controls type 1 error
while having good statistical power.

Post-hoc procedures
and violations of
assumptions
What if between the groups the assumptions are
not met (normality, equality of variance)?
Most multiple comparison procedures tolerate
small deviations from normality
They do not tolerate unequal group sizes and
different population variances
SPSS has 4 tests for dealing with different
population variances: Tamhane's T2, Dunnett's
T3, Games-Howell, Dunnet's C

Guidelines for post hoc comparisons
When the sample sizes are equal and population
variances similar: REGWQ or Tukey
Take Bonferroni if you want to cotrol Type I error
tightly
When sample sizes differ: Gabriel's procedure
When equality of variances is doubtful: GamesHowell

Running a one-way ANOVA on SPSS
(using viagra.sav)
The data are organized in columns:
'dose' codes the independent variable
(1=placebo; 2=low dose; 3=high dose)
 libido is the dependent variable

This coding looks
different from the
dummy coding
scheme in the
regression analysis.
However, the
dummy codes will
be used in the
contrasts!
The one-way ANOVA tests whether there is a
difference between the 3 group means of the
dependent variable (libido), as manipulated by
your independent variable (dose)
Analyze --> Compare Means --> One-way ANOVA
(using viagra.sav)
Dependent Variable
Factor =
Independent Variable
Dependent Variable: Libido
Caution: In an ANOVA you have only 1 dependent
variable (SPSS suggests various ones. For
analysing > 1 dependent variables you have to
use a Multiple ANOVA (MANOVA)
Independent Variable ('Factor'): Dose
Planned comparisons in SPSS
1. Contrasts: In the dialog box “Contrasts“ specify
the kind of contrast you want. We want to check if
there is a polynomial trend across the 3 groups
and thus tick 'Polynomial' and specify 'Quadratic'
(which is the highest trend there can be for
3 groups).
2. Planned comparisons:
Specifying the contrasts
1st contrast
placebo vs. Low and high
dose
2nd contrast
Low vs high dose
Next 
Product
DV 1
DV2
Contrast1 x
Group
(contrast1) (contrast2) Contrast 2)
Placebo
-2
0
0
Low dose
1
-1
-1
High dose
1
1
1
Total
0
0
0
Post-hoc tests in SPSS
Bug in the book:
Specify “First”
Since we have equally
sized groups
For checking the
REGWQ and
Tukey
We expect:
2 exp groups
should be >
control group
Note: Normally, either you do contrasts OR posthoc tests, NOT both in the same analysis.
Options
Levene's test
checks the
assumption of
homogeneous
variances
between groups
Produces a graph
Note: The Options box in SPSS-10 (above) misses
'Fixed Random effects', 'Brown-Forsythe' and 'Welch'
which are present in SPSS-13.
'Brown-Forsythe' and 'Welch' are alternative F-ratios
when the assumption of homogeneous variances is
violated.
Output from one-way ANOVA
Descriptives: Gives you a summary of all
important descriptive values
Descriptives
LIBIDO Libido
N
1,00 Placebo
2,00 Low Dose
3,00 High Dose
Total
5
5
5
15
Mean
2,2000
3,2000
5,0000
3,4667
Std. Deviation
1,3038
1,3038
1,5811
1,7674
Std. Error
,5831
,5831
,7071
,4563
95% Confidence Interval for
Mean
Lower Bound Upper Bound
,5811
3,8189
1,5811
4,8189
3,0368
6,9632
2,4879
4,4454
Minimum
1,00
2,00
3,00
1,00
Maximum
4,00
5,00
7,00
7,00
Error bar chart
Graphs Error bar, simple
Define:
Choose either SD or SE
Error bars represent SD
Error bars represent SE: no
overlap between placebo and
high dose  * difference
Homogeneity of variances
Test of Homogeneity of Variances
LIBIDO Libido
Levene
Statistic
df1
,092
2
df2
12
Sig.
,913
Levene's test is n.s., i.e.,
the three group do not have
differerent variances
ANOVA summary table
Effects due to
the model
Can the data be
accomodated
by a linear
model?
Experimental
effect
Mean Squares of
the Model MSM
ANOVA
The linear
Model (as well as
the linear Model
+ the quadratic
trend) can explain
the data
sufficiently
LIBIDO Libido
Between
Groups
(Combined)
Linear Term
Contrast
Deviation
Quadratic Term
Contrast
Within Groups
Total
Unsystematic variance
Sum of
Squares
20,133
19,600
2
1
Mean Square
10,067
19,600
F
5,119
9,966
Sig.
,025
,008
,533
1
,533
,271
,612
,533
23,600
43,733
1
12
14
,533
1,967
,271
,612
Total Sum
of Squares SST
df
Residual Sum
of Squares SSR
Trend analysis:
Is there a quadratic
trend in the data?
**
ns
MSTrend
No quadratic trend
Mean Squares of
the Residual
MSR
Result
There is a significant effect of the experimental
conditions (placebo, low, high dose)
By looking at the error bars representing SD's,
we would not have been able to tell. However, if
we take error bars to represent SE's, there is no
overlap between placebo and high dose. This
corresponds to the significant overall effect.

Aside: linear term + quadratic trend
How does the linear term and the quadratic term add in
the ANOVA?
SS Linear term:
SS Quadratic term:
SS combined
19.6
0.533
20.133
The quadratic trend analysis seeks for a non-linear trend
in the data, e.g., for a 'U'- or 'inverted 'U'-shaped curve
(see pictures). However, there is none (SSQ-Trend =
0.533). The linear SSM (19.6) is much bigger, hence a
linear Model fits the data. The linear term alone can
explain more variance (MSM = 19.6) than the combined
term (MSCombined = 10.067), however, both together are
still significant.
Output for 'planned comparisons'
1st contrast: placebo (-2) vs. both exp. groups
(Low dose (1) and high dose (1))
2nd contrast: low dose (-1) against high dose (1)

Under an undirected
hypothesis (two-tailed)
only the 1st contrast
is * (0.029)
Under a directed
hypothesis (1-tailed):
The 1st contrast is *
The 2nd contrast is *
(0.029/2=0.0145;
0.065/2=0.0325)
Contrast Coefficients
Contrast
1
2
DOSE Dose of Viagra
2,00 Low
3,00 High
1,00 Placebo
Dose
Dose
-2
1
1
0
-1
1
Weighted sum of group means:
group mean x weight of
the contrast
Contrast Tests
LIBIDO Libido
Assume equal variances
Does not assume equal
variances
Look here in case of
unequal variances
Contrast
1
2
1
2
Value of
Contrast
3,8000
1,8000
3,8000
Std. Error
1,5362
,8869
1,4832
t
2,474
2,029
2,562
df
12
12
8,740
Sig. (2-tailed)
,029
,065
,031
1,8000
,9165
1,964
7,720
,086
For nerds: value of contrasts
For the values of contrast, you sum up the weighted
means of the groups which figure in the contrast:
1st contrast:
(XW) = [(2.2 x -2) + (3.2 x 1) + (5 x 1)] = 3.8
2nd contrast:
(XW) = [(3.2 x -1) + (5 x 1)] = 1.8
Summary ANOVA
1st contrast: There is a general effect of the
experimental treatment: taking viagra increases
the libido
2nd contrast: There is a special effect of the two
experimental groups: Taking a high dose of viagra
increases the libido more than taking a low dose
Output for post hoc tests
Each group is compaired with each other. We had
requested 3 tests: Tukey, Games-Howell, and Dunnett.
Multiple Comparisons
Dependent Variable: LIBIDO Libido
Tukey HSD
(I) DOSE Dose of Viagra (J) DOSE Dose of Viagra
1,00 Placebo
2,00 Low Dose
2,00 Low Dose
Games-Howell
3,00 High Dose
1,00 Placebo
3,00 High Dose
Mean
Difference
(I-J)
Std. Error
-1,0000
,8869
-2,8000*
,8869
1,0000
,8869
-1,8000
,8869
Sig.
ns ,516
* ,021
ns ,516
,147
95% Confidence Interval
Lower Bound Upper Bound
-3,3663
1,3663
-5,1663
-,4337
-1,3663
3,3663
-4,1663
,5663
3,00 High Dose
1,00 Placebo
2,00 Low Dose
2,8000*
1,8000
,8869
,8869
,021
,147
,4337
-,5663
5,1663
4,1663
1,00 Placebo
2,00 Low Dose
3,00 High Dose
-1,0000
-2,8000*
,8869
,8869
-3,3563
-5,4390
1,3563
-,1610
2,00 Low Dose
1,00 Placebo
3,00 High Dose
1,0000
-1,8000
,8869
,8869
ns ,479
* ,039
,479
ns ,185
-1,3563
-4,4390
3,3563
,8390
3,00 High Dose
1,00 Placebo
2,00 Low Dose
2,8000*
1,8000
,8869
,8869
,039
,185
,1610
-,8390
5,4390
4,4390
1,00 Placebo
1,00 Placebo
1,0000
2,8000*
,8869
,8869
ns ,227
* ,008
Dunnett t (<control)a 2,00 Low Dose
3,00 High Dose
Note: in our
planned
comparisons
this contrast
was *!
2,8698
4,6698
*. The mean difference is significant at the .05 level.
a. Dunnett t-tests treat one group as a control, and compare all other groups against it.
Why are some contrasts ns for post hoc tests but * for planned comparisons?
Post hoc tests are always 2-sided (you do not have any specific hypothesis);
except Dunnett test where we did a 1-tailed test
Post hoc tests are always conservative in that they try to avoid inflation of
familywise error
Homogeneous subsets
LIBIDO Libido
Tukey HSDa
Ryan-Einot-Gabri
el-Welsch Range
DOSE Dose of Viagra
1,00 Placebo
2,00 Low Dose
3,00 High Dose
Sig.
1,00 Placebo
2,00 Low Dose
3,00 High Dose
Sig.
N
5
5
5
5
5
5
Subset for alpha = .05
1
2
2,2000
3,2000
3,2000
5,0000
,516
,147
2,2000
3,2000
3,2000
5,0000
,282
,065
Means for groups in homogeneous subsets are displayed.
a. Uses Harmonic Mean Sample Size = 5,000.
SPSS creates subsets
of groups with similar
means, here placebo +
low dose (1) and low
dose + high dose (2).
Within those groups,
there is ns difference
Calculating the effect size (overall ANOVA)
Since SPSS does not routinely caculate the effect size,
we have to do it ourselves.
Since we know R2, we can derive r (effect size) from it:
R2 = SSM = 20.13 = .46
SST
43.73
r =  .46
= .68
Since .68 is > .5, the benchmark of a big effect, we can
safely claim that our finding is a substantial one.
An alternative and more exact measure is  which
estimates the effect size in the population. (= .60).
Calculating the effect size (single contrasts)
rcontrasts =
t
rcontrast 1 =
2.474
/ 2.4742 + 12 = .58 big effect
rcontrast 2 =
2.029
/ 2.0202 + 12 = .51 big effect
2
/t2 + df
2
2
Reporting 1-way ANOVA ( Field, 2005, p 359)
There was a significant effect of Viagra on levels
of libido, F (2,12) = 5.12, p < .05,  = .60.
There was a significant linear trend, F (1,12) =
9.97, p < .01, = .62, indicating that as the dose
of Viagra increased, libido increased
proportionately.
Planned contrasts revealed that having any dose
of Viagra significantly increased libido compared
to having a placebo, t (12) = 2.47, p < .05 (onetailed), r = .58, and that having a high dose
significantly increased libido compared to having a
low dose, t (12) = 2.03, p < .05 (one-tailed), r =
.51.
