Download Example Factor Analysis Questions

ANCOVA and MANCOVA: Example Questions and Outline Model Answers
Question 1
A double-blind, placebo-controlled trial was carried out to test a new drug which is supposed to improve episodic memory in healthy elderly
subjects, many of whom show mild episodic memory impairment. Before starting the trial the verbal IQ of the subjects was assessed. The
subjects were then assigned at random to an active drug or placebo treatment. After completing the treatment the researchers administered 3
standard verbal episodic memory tests to each subject. They felt that each test was measuring the memory processes that the drug was likely to
improve. After breaking the drug codes, the researchers used ANOVAs to see if the post-treatment memory scores were higher in the drugtreated subjects. The results are shown in Part I of the SPSS printout. Disappointed that only one memory test appeared to show a significant
drug effect, the researchers consulted colleagues. After looking at the correlations (printout Part II) between the memory test performance and
pretreatment verbal IQ, one colleague suggested that they repeat the analyses covarying out the verbal IQ scores, stressing that they should test
the homogeneity of regressions first. The homogeneity of regression and ANCOVAs (printouts Part III and IV respectively) were as shown.
(a)
Explain what the homogeneity of regression printout shows and say, in light of these results, whether the ensuing ANCOVAs were
justified. (30% of marks for question)
(b)
Briefly explain the purpose of the ANCOVAs and why the ANCOVA results show stronger drug effects than the earlier ANOVAs. (35%
of marks for question)
(c)
The researchers concluded that 2 of their 3 memory tests were significantly improved by the drug after removing the effect of baseline
verbal IQ. What problem means that they were not justified in making this claim? (20% of marks for question)
(d)
Another colleague suggested an alternative, but related, analytic method which avoided the problem in (c). This technique showed a
significant drug effect on episodic memory test performance after removing the effect of baseline verbal IQ. Name the technique and
briefly note why it is preferable here? (15% of marks for question)
Printout for Question 1
Part I – Anova Results + Means
Tests of Between-Subjects Effects
Source
TREATMNT
Error
Dependent Variable
Memory tes t 1
Memory tes t 2
Memory tes t 3
Memory tes t 1
Memory tes t 2
Memory tes t 3
Type III Sum
of Squares
16.240
15.123
7.516
294.049
151.702
447.413
df
1
1
1
58
58
58
Mean Square
16.240
15.123
7.516
5.070
2.616
7.714
Part II – Correlation Matrix
Correlations
Memory tes t 1
Memory tes t 2
Memory tes t 3
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Memory tes t 2
.434
.001
60
Memory tes t 3
.758
.000
60
.344
.007
60
Verbal IQ
.652
.000
60
.388
.002
60
.353
.006
60
F
3.203
5.782
.974
Sig.
.079
.019
.328
Part III – Homogeneity of Regression Results
Tests of Between-Subjects Effects
Source
VERBIQ
TREATMNT * VERBIQ
Error
Dependent Variable
Memory tes t 1
Memory tes t 2
Memory tes t 3
Memory tes t 1
Memory tes t 2
Memory tes t 3
Memory tes t 1
Memory tes t 2
Memory tes t 3
Type III Sum
of Squares
127.211
26.396
55.744
4.338E-03
3.475
.154
164.100
123.962
391.240
df
1
1
1
1
1
1
56
56
56
Mean Square
127.211
26.396
55.744
4.338E-03
3.475
.154
2.930
2.214
6.986
F
43.412
11.925
7.979
.001
1.570
.022
Sig.
.000
.001
.007
.969
.215
.882
Part IV – Ancova Results and Covariate-Adjusted Means
Tests of Between-Subjects Effects
Source
VERBIQ
TREATMNT
Error
Dependent Variable
Memory tes t 1
Memory tes t 2
Memory tes t 3
Memory tes t 1
Memory tes t 2
Memory tes t 3
Memory tes t 1
Memory tes t 2
Memory tes t 3
Type III Sum
of Squares
129.945
24.265
56.018
14.396
14.337
6.691
164.104
127.437
391.394
df
1
1
1
1
1
1
57
57
57
Mean Square
129.945
24.265
56.018
14.396
14.337
6.691
2.879
2.236
6.867
F
45.135
10.853
8.158
5.000
6.413
.974
Treatment Condition
Dependent Variable
Memory tes t 1
Memory tes t 2
Memory tes t 3
Treatment Condition
active drug
placebo
active drug
placebo
active drug
placebo
Mean
8.217a
7.237a
12.316 a
11.338 a
9.190a
8.522a
Std. Error
.310
.310
.273
.273
.478
.478
a. Evaluated at covariates appeared in the model: Verbal IQ = 92.68.
95% Confidence Interval
Lower Bound Upper Bound
7.597
8.838
6.617
7.858
11.770
12.863
10.792
11.885
8.232
10.148
7.564
9.480
Sig.
.000
.002
.006
.029
.014
.328
Question1: MODEL ANSWER
Note how short the perfect answer can be…
(a)
(b)
(c)
(d)
The homogeneity of regression printout shows non-significant covariate(IQ)*treatment group interactions for each of the 3 memory tests
used. This means that the regressions of IQ on the dependent variable (i.e. each of the memory test scores) are statistically
indistinguishable across the levels of the group (i.e. treatment) factor. (Students might draw diag with parallel regression lines to illustrate
this -- must be properly labelled; they do not need to draw a diag if their verbal description is complete and clear.) Thus the regressions are
homogeneous and mean that ANCOVA’s requirement for homogeneous regressions is satisfied for each of the 3 DVs. In terms of this
requirement it is legitimiate to carry out the 3 ANCOVAs (on each of the memory tests as separate DVs) on these data.
The purpose of the ANCOVA is to reduce the size of the error term in the analyses thereby increasing power. It achieves this by removing
a source of variation from the DV (the DV’s covariance with the covariate). This source of variation is normally included in the error term
of the ANOVA. The F-ratio of the ANOVA (or ANCOVA) is formed from an effect term divided by the error term: if the error term is
reduced by assigning some of its variance to a covariate, then the F-ratio for the effect of interest increases commensurately. [A student
who says that the ANCOVA is to remove any IQ differences between the drug-treatment groups gets ABSOLUTELY ZERO marks for this
part.] For memory test 1, which has by far the strongest correlation with IQ (r>0.65), we can see this in operation ANOVA:
MS(effect)=16.2; MS(error)=5.070. ANCOVA: MS(effect)=14.4; MS(error)=2.879. For the other two memory tests (with lower
correlations with IQ; r<0.4), the effect of covariate adjustment was very modest, barely affecting the F-ratios.
The researchers have capitalised on chance by conducting 3 separate tests of their hypothesis. None of these individual tests will therefore
truly have a Type I error rate of 5% and so cannot be judged by this criterion. If one adopted a Bonferroni correction to deal with this, the
necessary alpha level would be 0.05/3. (i.e. 0.0167) and only memory test 2 (after covariate adjustment) would be deemed significant.
MANCOVA (would accept MANOVA; although strictly it is MANCOVA if IQ is included as a covariate). Basically, it is preferable
because it uses all 3 memory test DVs simultaneously to carry out a single test of the researchers’ hypothesis rather than three separate
tests. It is appropriate because the 3 tests are thought to be measuring the same processes and the correlations between the 3 DVs are
neither close to zero or 1 (they were 0.34; 0.43; 0.76).