Download Chapter 10

Chapter 10 - Part 1
Factorial Experiments
Nomenclature
• We will use the terms Factor and Independent
Variable interchangeably. They mean the same
thing.
• The term “factorial analysis of variance” simply
means the analysis of variance when there are
multiple factors (multiple independent variables.)
• I will sometimes use the phrase Factor 1 or 2
interchangeably with Independent Variable 1 or 2.
But to prevent confusion, whenever the term is
abbreviated I will use IV1 or IV2, not F1 or F2.
Two-way Factorial Experiments
In Chapter 10, we are studying experiments
with two independent variables, each of
which will have multiple levels.
We call each independent variable a factor. The
first IV is called Factor 1 or IV1. The second
IV is called Factor 2 or IV2.
So, in this chapter we will study two factor,
unrelated groups experiments.
Conceptual Overview
• In Chapter 9 you learned to do the F test
comparing two estimates of sigma2, MSB and
MSW. That is what you do in simple experiments,
those with only one independent variable.
• In the single IV, unrelated groups experiments in
Ch. 9, F = MSB/MSW
• That is one version of a more generic formula. The
generic formula tells us what to do in the two (or
more) factor case.
• Here is the basis for the generic formula
Generic formula for the unrelated
groups F test
• F = SAMP.FLUC. + (?) ONE SOURCE OF VARIANCE)
(SAMPLING FLUCTUATION)
• Which can also be written as
• F = (ID + MP + (?) ONE SOURCE OF VARIANCE)
(ID+ MP)
Let me explain why.
The denominator of the F test
• The denominator in the F test reflects variation within
each group because of random individual differences (ID)
and measurement problems (MP). Since everyone in the
same group is treated the same, only ID + MP can
contribute to within group variation. So, MSW can not
reflect the effect of any independent variable or
combination of independent variables.
• Random sampling fluctuation is based on random
individual differences and measurement problems (ID +
MP).
• Thus, in the unrelated groups F and t test, our best index of
the random effects of individual differences and
measurement problems, the basis of random sampling
fluctuation, is MSW, our least squares, unbiased estimate of
sigma2.
Denominator = MSW
• In the F tests we are doing, F tests for unrelated
groups, MSW serves as our best estimate of
sigma2.
• To repeat, in computing MSW, we compare each
score to the mean of its own, specific group.
Everyone in each specific group is treated the
same.
• So, the only reasons that scores differ from the
other scores in their group and their own group
means is that people differ from each other (ID)
and there are always measurement problems (MP).
• MSW = ID +MP
Numerator of the F ratio:
Generic formula
• Numerator of the F ratio is an estimate of sigma2
that reflects sampling fluctuation + the possible
effects of one difference between the groups.
• In Ch. 9, there was only one difference among the
ways the groups were treated, the different levels
of the independent variable (IV)
• MSB reflected the effects of random individual
differences (there are different people in each
group), random measurement problems, and the
effects of the independent variable.In Ch. 9, we
could write that as MSB = ID + MP + (?)IV
To repeat, in the single factor
analysis of variance
F = (ID + MP + ?IV)
(ID + MP)
• Both the numerator and denominator reflect the
same elements underlying sampling fluctuation
• The numerator includes one, and only one,
systematic source of variation not found in the
denominator.
This allows us to conclude that:
• IF THE NUMERATOR IS SIGNIFICANTLY
LARGER THAN THE DENOMINATOR, THE SIZE
DIFFERENCE MUST BE MUST BE CAUSED BY
THE ONE ADDITIONAL THING PUSHING THE
MEANS APART, the IV.
• But notice there can’t be more
than one thing in the numerator
that does not appear in the
denominator to make that
conclusion inevitable.
Why we can’t use MSB as the numerator in the
multifactor analysis of variance
•
In the two factor analysis of variance, the
means can be pushed apart by:
–
–
–
–
The effects of the first independent variable
(IV1).
The effects of the second independent
variable (IV2)
The effects of combining IV1 and IV2 that are
above and beyond the effects of either
variable considered alone (INT)
Random sampling fluctuation (ID + MP)
So if we compared MSW to MSB in a two factor
experiment, here is what we would have.
• F = (ID + MP + ?IV1 + ?IV2 + ?INT)
(ID + MP)
That’s not an F test. In an F test the numerator must
have one and only one source of variation beyond
sampling fluctuation. HERE THERE ARE
THREE OF THEM!
Each of these three things besides sampling
fluctuation could be pushing the means apart.
So, the F ratio would be meaningless.
To use the F test with a two
factor design, we must create 3
numerators to compare to MSW,
each comprising ID + MP + one
other factor.
HOW?
To obtain our 3 numerators for the F
test, we divide (analyze) the sums of
squares and degrees of freedom
between groups (SSB & dfB) into
component parts. Each part must
contain only one factor along with ID
and MP. Then each component will
yield an estimate of sigma2 that can
be compared to MSW in an F test.
Write out the answer to these two
questions without reading the
answer from the slides:
• Why can’t you compare MSB to MSW in the
two factor, unrelated groups F test?
• What must you do instead?
ANSWER 1: If we compared MSW to MSB in a two
factor experiment, here is what we would have.
• F = (ID + MP + ?IV1 + ?IV2 + ?INT)
(ID + MP)
That’s not an F test. In an F test the numerator must
have one and only one source of variation beyond
sampling fluctuation. HERE THERE ARE
THREE OF THEM!
Each of these three things besides sampling
fluctuation could be pushing the means apart.
So, the F ratio would be meaningless.
ANSWER 2:We must take apart
(analyze) the sums of squares and
degrees of freedom between groups
(SSB & dfB) into component parts.
Each part must contain only one
factor along with ID and MP. Then
each component will yield an
estimate of sigma2 that can be
compared to MSW in an F test.
Here is how we divide SSB and dfB
into their component parts:
• First, we create a way to study the effects of factor
1 alone.
• To do that, we combine groups so that the
resulting, larger aggregates of participants differ
only because they received different levels of the
first independent variable, IV1.
• Each such combined group will include an equal
number of people who received the different levels
of IV2.
• So the groups are the same in that regard.
• They differ only on how they were treated on the
first independent variable, IV1.
Computing MSIV1, one of the three
numerators in a two factor F test
• If we find the differences between each person’s
combined group mean and the overall mean,
square and sum them, we will have a sum of
squares for the first independent variable (SSIV1).
• Call the number of levels of an independent
variable L. df for the combined group equals the
number of levels of its IV minus one (LIV – 1).
• An estimate of sigma2 that includes only ID + MP
+ (?) IV1 can be computed by dividing this sum of
squares by its degrees of freedom, as usual.
• MSIV1 = SSIV1/dfIV1 = (ID + MP + ?IV1)
Once you have MSIV1, you have
one of the three F tests you do in
a two factor ANOVA
• F = MSIV1/MSW
Then you do the same thing to
find MSIV2
• You combine groups so that you have groups that
differ only on IV2.
• You compare each person’s mean for this
combined group to the overall mean, squaring athe
differences for each person and then summing
them for the entire sample to get SSIV2.
• Degrees of freedom = the number of levels of
Factor 2 minus 1 (dfIV2 = LIV2 – 1).
• Then MSIV2 = SSIV2/dfIV2
• FIV2 = MSIV2/MSW
What’s left is the interaction.
• Remember, we are subdividing SSB and dfB into their
three component parts.
• We have already computed SSIV1, SSIV2, dfIV1, and dfIV2.
• WHAT’S LEFT? The part of SSB and dfB that hasn’t been
accounted for is the sum of squares and degrees of
freedom for the interaction.
• The interaction involves the means being pushed apart by
the two independent variables having a different effect
when present together than either has by itself alone.
• For example, a moderate dose of alcohol can make you
intoxicated. A moderate dose of barbituates can make you
sleepy. Taken together they multiply each others’ effects
and the interaction of the two drugs can easily make you
dead.
How to compute the interaction.
• To compute the sum of squares and df for the interaction,
we find that part of the sum of squares between groups
and degrees of freedom between groups that are not
accounted for by Factor 1 (IV1) and Factor 2 (IV2)
• THAT IS, Y0U SUBTRACT THE SUMS OF SQUARES AND
df YOU’VE ALREADY ACCOUNTED FOR (SSIV1, SSIV2,
dfIV1, and dfIV2) FROM THE SUM OF SQUARES AND
DEGREES OF FREEDOM BETWEEN GROUPS (SSB & dfB).
• WHAT’S LEFT IS SSINT & dfINT, THE SUM OF
SQUARES AND DEGREES OF FREEDOM FOR THE
INTERACTION.
Look at that another way: The whole
is equal to the sum of its parts.
• SSB and dfB are the between group sum of
squares and degrees of freedom. Each is composed
of three parts, SSIV1, SSIV2, SSINT and dfIV1, dfIV2,
and dfINT.
• So if we subtract the SS and df for factors 1 & 2
from SSB and dfB, what is left is the sum of
squares and df for the interaction.
• SSINT = SSB-(SSIV1+ SSIV2)=SSB- SSIV1- SSIV2
• dfINT = dfB-(dfIV1+ dfIV2)=dfB- dfIV1- dfIV2
Designs of 2 factor studies
Analysis of Variance
• Each possible combination of IV1 and IV2 creates
an experimental group. Participants are randomly
assigned to each of the treatment groups.
• Each experimental group is treated differently
from all other groups in terms of one or both
factors.
• For example, if there are 2 levels of the first
variable (Factor 1or IV1) and 2 of the second
(IV2), we will need to create 4 groups (2x2). If IV1
has 2 levels and IV2 has 3 levels, we need to create
6 groups (2x3). If IV1 has 3 levels and IV2 has 3
levels, we need 9 groups. Etc.
Some nomenclature
• Two factor designs are identified by simply stating
the number of levels of each variable. So a 2x4
design (called “a 2 by 4 design”) has 2 levels of
IV1 and 4 levels of IV2. A 3x2 design has 3 levels
of IV1 and 2 levels of IV2. And so on.
• Which factor is called IV1 and which is called IV2
is arbitrary (and up to the experimenter).
Example of 2 x 3 design
To make it more concrete, assume we are testing new
treatments for Generalized Anxiety Disorder. In a two
factor design we examine the effects of cognitive behavior
therapy vs. a social support group among GAD patients
who receive Ativan, Zoloft or Placebo. Thus, IV1 has 2
levels (CBT/Social Support) while IV2 has 3 levels
(Ativan/Zoloft/Placebo) So, we would form 2 x 3 = 6
groups to do this experiment. Half the patients would get
CBT, the other half get social support. A third of the CBT
patients and one-third of the Social Support patients also
get Ativan. Another third of those who receive CBT and
one-third of those who get social support also receive
Zoloft. The final third in each psychotherapy condition get
a pill placebo.
A 2 x 3 design yields 6 groups. Let’s say
you have 24 participants. Four are
randomly assigned to each group.
• Here are the six treatment groups:
–
–
–
–
–
–
CBT + Ativan
CBT + Zoloft
CBT + Placebo
Social support + Ativan
Social support + Zoloft
Social support + Placebo
Example: Experiment Outline
• Population: Outpatients with Generalized Anxiety Disorder
• Subjects: 24 participants divided equally among 6 treatment
groups.
• Independent Variables:
– Factor 1: Psychotherapy: CBT or Social Support (SoSp)
– Factor 2: Medication: Ativan, Zoloft, or Placebo
• Groups: 1=CBT + Ativan; 2=CBT + Zoloft; 3=CBT +
Placebo; 4=SoSp + Ativan; 5=SoSp + Zoloft; 6=SoSp +
Placebo.
• Dependent variable: Anxiety remaining after treatment.
Lower scores equal less anxiety and a better outcome.
A 3X2 STUDY
Type of drug
CBT
Ativan
Zoloft
Placebo
X CBTA
X CBTZ
X CBTP
X SoSpA
X SoSpZ
X SoSpP X SoSp
X Ativan
X Zoloft
X Placebo M
X CBT
Type of
therapy
SoSp
Effects
• We are interested in the main effects of type of
psychotherapy and type of drug. Do participants
get better with CBT and not with SoSp or the
reverse? Do people get better when they get a mild
tranquilizer (Ativan) an SSRI (Zoloft) or Placebo.
• We are also interested in assessing how combining
different levels of both factors affect the response
in ways beyond those that can be predicted by
considering the effects of each IV separately. So,
we are interested in the interaction of the
independent variables.
MSW
Ativan
CBT
Drug Given
Zoloft
Compare each score
to the mean for its group.
XX
Placebo
X CBTA
X CBTZ
X CBTP
X CBT
X SoSpA
X SoSpZ
X SoSpP
X SoSp
X Ativan
X Zoloft
X Placebo M
Type of
therapy
SoSp
Mean Squares Within Groups
S#
1.1
1.2
1.3
1.4
X
3
4
8
9
X
6
6
6
6
2.1
2.2
2.3
2.4
7
3
4
2
3.1
3.2
3.3
3.4
15
14
16
11
X  X  X  X 
-3
9
2
-2
2
3
4
4
9
S#
4.1
4.2
4.3
4.4
4
4
4
4
3
-1
0
-2
9
1
0
4
5.1
5.2
5.3
5.4
6
9
12
13
14
14
14
14
1
0
2
-3
1
0
4
9
6.1
6.2
6.3
6.4
14
18
19
21
X
12
7
7
6
X
8
8
8
8
X  X  X  X 
-4
16
2
-1
-1
-2
1
1
4
10
10
10
10
-4
-1
2
3
16
1
4
9
18
18
18
18
-4
0
1
3
16
0
1
9
dfW  n  k  24  6  18
MSW
• SSw = 132.00
• dfW = 18
• MSW = 132/18 = 7.33
Then we compute a sum of
squares and df between groups
• This is the same as in Chapter 9
• The difference is that we are going to
subdivide SSB and dfB into component
parts.
• Thus, we don’t use SSB and dfB in our
Anova summary table, rather we use them
in an intermediate calculation.
Compare each group mean
to the overall mean.
Sum of Squares
Between Groups (SSB)
X M
Type of Drug
Ativan
CBT
Zoloft
Placebo
X CBTA
X CBTZ
X CBTP
X CBT
X SoSpA
X SoSpZ
X SoSpP
X SoSp
X Ativan
X Zoloft
X Placebo M
Type of
Therapy
SoSp
Means for GAD study
Type of Drug
Ativan
Cognitive BT
Zoloft
Placebo
6
4
14
8.00
8
10
18
12.00
7.00
7.00
Diet Type
Social Support
16.00
M  10.00
Sum of Squares Between Groups
(M=10.00, SSB=544 , dfB=5)
X
6
6
6
6
M
10
10
10
10
4
4
4
4
14
14
14
14
X  M  X  M 
2
-4
-4
-4
-4
16
16
16
16
X
8
8
8
8
10
10
10
10
-6
-6
-6
-6
36
36
36
36
10
10
10
10
10
10
10
10
4
4
4
4
16
16
16
16
18
18
18
18
M
10
10
10
10
X  M  X  M 
2
-2
-2
-2
-2
4
4
4
4
10
10
10
10
0
0
0
0
0
0
0
0
10
10
10
10
8
8
8
8
64
64
64
64
df B  k 1  6 1  5
Next, we answer the questions about each
factor having an overall effect.
• To get proper between groups mean squares we
have to divide the sums of squares and df between
groups into components for factor 1, factor 2, and
the interaction.
• Let’s just look at factor 1. Our question about
factor 1 was “Do people undergoing different
therapies have differential responses to any task?”
• We can group participants into all those who were
treated with CBT and those treated with Social
Support.
Forming Groups that Differ only on Factor 1
• Pretend that the experiment was a simple, single
factor experiment in which the only difference
among the groups was the first factor (that is, the
type of therapy given each group). Create groups
reflecting only differences on Factor 1.
• So, when computing the main effect of Factor 1 (type
of psychotherapy), ignore Factor 2 (type of drug).
Divide participants into two groups depending solely
on whether they were given CBT or Social Support.
Then, find the mean of each of the two combined
groups (CBT and Social Support).
Computing SS for Factor 1
Next, find the deviation of the mean of the CBT
participants from the overall mean. Then sum
and square those differences.
Then, find the deviation of the mean of the
Social Support participants from the overall
mean. Then sum and square those differences.
The total of the summed and squared deviations
the mean of each of the combined groups from
M, the overall mean, is the sum of squares for
Factor 1. (SSIV1).
dfIV1 and MSIV1
• Compute a mean square that takes only differences
on Factor 1 into account by dividing SSIV1 by
dfIV1.
• For example, in this experiment, type of therapy
was either CBT or Social Support. The two ways
participants are treated are called the two “levels”
of Factor 1.
• dfIV1= L1 – 1 = 2-1 = 1 [where L1 equals the
number of levels (or different variations) of the
first factor (IV1)].
SSIV1: Main Effect
of Therapy Type
Type of drug
CBT
Compare each score’s therapy
mean to the overall mean.
X M
Ativan
Zoloft
Placebo
X CBTA
X CBTZ
X CBTP
X CBT
X SoSpA
X SoSpZ
X SoSpP
X SoSp
X Ativan
X Zoloft
X Placebo M
Therapy
Type
SoSp
Means for GAD study
Type of New Drug
Ativan
Cognitive BT
Zoloft
Placebo
6
4
14
8.00
8
10
18
12.00
7.00
7.00
Diet Type
Social Support
16.00
M  10.00
CBT/SoSp Means, M=10.00
Means: CBT=8.00; SoSp=12.00
S#
X
CBT group
1.1
8
1.2
8
1.3
8
1.4
8
2.1
8
2.2
8
2.3
8
2.4
8
3.1
8
3.2
8
3.3
8
3.4
8
n  12
S#
X
SoSp group
4.1 12
4.2 12
4.3 12
4.4 12
5.1 12
5.2 12
5.3 12
5.4 12
6.1 12
6.2 12
6.3 12
6.4 12
n  12
SSIV1=96.00; dfIV1=2-1=1;
MSIV1=96.00
X
8
8
8
8
8
8
8
8
8
8
8
8
M
10
10
10
10
10
10
10
10
10
10
10
10
X  M  X  M 
2
-2
-2
-2
-2
-2
-2
-2
-2
-2
-2
-2
-2
4
4
4
4
4
4
4
4
4
4
4
4
X
12
12
12
12
12
12
12
12
12
12
12
12
M
10
10
10
10
10
10
10
10
10
10
10
10
X  M  X  M 
2
2
2
2
2
2
2
2
2
2
2
2
2
4
4
4
4
4
4
4
4
4
4
4
4
Factor 2
Then pretend that the experiment was a single
experiment with only the second factor.
Proceed as you just did for
Factor 1 and obtain SSIV2 and MSIV2 where
dfIV2=LIV2 – 1=3-1=2.
SSIV1: Main Effect
of Drug Type
Drug Type
CBT
Therapy
Type
SoSp
Compare each score’s
Drug Type mean
to the overall mean.
X M
Ativan
Zoloft
Placebo
X CBTA
X CBTZ
X CBTP
X CBT
X SoSpA
X SoSpZ
X SoSpP
X SoSp
X Ativan
X Zoloft
X Placebo M
Means for GAD study
Type of New Drug
Ativan
Cognitive BT
Zoloft
Placebo
6
4
14
8.00
8
10
18
12.00
7.00
7.00
Diet Type
Social Support
16.00
M  10.00
Means: Ativan=7.00, Zoloft=7.00,
Placebo = 16.00, M=10.00
S# X
1.1
1.2
1.3
1.4
4.1
4.2
4.3
4.4
7
7
7
7
7
7
7
7
n=8
S# X
2.1 7
S# X
3.1 16
2.2
2.3
2.4
5.1
5.2
5.3
5.4
3.2
3.3
3.4
6.1
6.2
6.3
6.4
7
7
7
7
7
7
7
n=8
16
16
16
16
16
16
16
n=8
SSIV2=442; dfIV2=2; MSIV2=221
#S
Ativan
1.1
1.2
1.3
1.4
4.1
4.2
4.3
4.4
X
.
7
7
7
7
7
7
7
7
#S
X
Mild Emb.
2.1
7
2.2
7
2.3
7
2.4
7
5.1
7
5.2
7
5.3
7
5.4
5
M
10
10
10
10
10
10
10
10
M
10
10
10
10
10
10
10
10
 M  X
-3
-3
-3
-3
-3
-3
-3
-3
 M  X
-3
-3
-3
-3
-3
-3
-3
-3
 M  X
2
9
9
9
9
9
9
9
9
 M  X
#S
X
No Emb.
3.1
16
3.2
16
3.3
16
3.4
16
6.1
16
6.2
16
6.3
16
6.4
16
M
10
10
10
10
10
10
10
10
 M  X
6
6
6
6
6
6
6
6
 M  X
2
36
36
36
36
36
36
36
36
2
9
9
9
9
9
9
9
9
df E  LF1 1  3 1  2
Computing the sum of squares and df
for the interaction.
• SSB contains all the possible effects of the independent
variables in addition to the random factors, ID and MP. Here
is that statement in equation form
• SSB= SSIV1 + SSIV2 + SSINT
• Rearranging the terms:
SSINT = SSB - (SSIV1+SSIV2) or SSINT = SSB- SSIV1-SSIV2
SSINT is what’s left from the sum of squares between groups
(SSB) when the main effects of the two IVs are accounted for.
So, subtract SSIV1 and SSIV2 from SSB to obtain the sum
of squares for the interaction (SSINT).
Then, subtract dfIV1 and dfIV2 from dfB to obtain dfINT).
Means Squares for Interaction
MSINT
544  96  442  SS Interaction
REARRANGE
SS Interaction  544  96  442  6
SS Interaction  6
df Interaction  df B  df Embarrassment  df Difficulty  5  2 1  2  2
MS Interaction  6.00 / 2  3.00
Testing 3 null hypotheses in the
two way factorial Anova
Hypotheses for Therapy Type
• Null Hypothesis - H0: There is no effect of
Drug Type. The means for anxiety will be
the same whether the GAD patients were
given CBT or SocialSupport.
• Experimental Hypothesis - H1: The type of
psychotherapy a group gets, considered
alone, will affect their anxiety.
Hypotheses for Type of Drug
• Null Hypothesis - H0: The three drug types,
including the placebo, will have similar
effects on reported anxiety.
• Experimental Hypothesis - H1: The three
drug types, including the placebo, will have
different effects on reported anxiety.
Hypotheses for the Interaction of Type of
Therapy and Type of Drug
• Null Hypothesis - H0: There is no interaction effect.
Once you take into account the main effects of type
of therapy and type of drug, there will be no
differences among the groups that can not be
accounted for by sampling fluctuation.
• Experimental Hypothesis - H1: There are effects of
combining CBT and Social Support with specific
drugs can not be predicted from either IV considered
alone.
Computational steps
•
•
•
•
•
•
•
•
•
Outline the experiment.
Define the null and experimental hypotheses.
Compute the Mean Squares within groups.
Compute the Sum of Squares between groups.
Compute the main effects.
Compute the interaction.
Set up the ANOVA table.
Check the F table for significance.
Interpret the results.
Steps so far
•
•
•
•
•
•
Outline the experiment.
Define the null and experimental hypotheses.
Compute the Mean Squares within groups.
Compute the Sum of Squares between groups.
Compute the main effects.
Compute the interaction.
What we know to this point
• SSIV1=96.00, dfIV1=1; MSIV1=96.00
• SSIV2=442.00, dfIV2=2; MSIV2=221.00
• SSINT=6.00, dfINT=2; MSINT=3.00
• SSW=132.00, dfW=18; MSW=7.33
Steps remaining
• Set up the ANOVA table.
• Check the F table for significance.
• Interpret the results.
Therapy Type – F(1,18)=13.10, p<.01;
Drug Type – F(2,18)=30.14, p<.01
Interaction-F(2,18)=0.41, n.s.
SS
df
MS
F
p
Type of Therapy
96.00
1
96.00
13.10
.01
Type of Drug
442
2
221.00
30.14
.01
6
2
3.00
0.41
132
18
7.33
Interaction
Error
.05  3.55
.01  6.01
n.s.
State Results
• The main effects for CBT vs. Social Support was
significant– F(1,18)=13.10, p<.01.
• The main effect for Ativan vs. Zoloft vs. Placebo
was significant - F(2,18)=30.14, p<.01
• The interaction was not significant (F(2,18)=0.41,
n.s.)
Interpret Significant
Results
Describe pattern
of means.
• CBT participants were less anxious than
were those who received Social Support.
• Participants who received the two active
medications (Ativan and Zoloft had the
same mean anxiety ratings after treatment,
while those who received the placebo were
much more anxious.
Interpret Significant Results
• These findings are consistent an additive
effect of appropriate medication and CBT
for Generalized Anxiety Disorder. Both
lower anxiety among these patients.
Reconcile statistical findings
with the hypotheses.