Download i4fms234ti

Assessment 4 Context
Recall that null hypothesis tests are of two types: (1) differences between group means and (2)
association between variables. In both cases there is a null hypothesis and an alternative
hypothesis. In the group means test, the null hypothesis is that the two groups have equal
means, and the alternative hypothesis is that the two groups do not have equal means. In the
association between variables type of test, the null hypothesis is that the correlation coefficient
between the two variables is zero, and the alternative hypothesis is that the correlation
coefficient is not zero.
Notice in each case that the hypotheses are mutually exclusive. If the null is false, the
alternative must be true. The purpose of null hypothesis statistical tests is generally to show
that the null has a low probability of being true (the p value is less than .05) – low enough that
the researcher can legitimately claim it is false. The reason this is done is to support the
allegation that the alternative hypothesis is true.
In this context you will be studying the details of the first type of test again, with the added
capability of comparing the means among more than two group at a time. This is the same type
of test of difference between group means. In variations on this model, the groups can actually
be the same people under different conditions. The main idea is that several group mean values
are being compared. The groups each have an average score or mean on some variable. The
null hypothesis is that the difference between all the group means is zero. The alternative
hypothesis is that the difference between the means is not zero. Notice that if the null is false,
the alternative must be true. It is first instructive to consider some of the details of groups.
One might ask why we would not use multiple t tests in this situation. For instance, with three
groups, why would I not compare groups one and two with a t test, then compare groups one
and three, and then compare groups two and three?
The answer can be found in our basic probability review. We are concerned with the probability
of a TYPE I error (rejecting a true null hypothesis). We generally set an alpha level of .05, which
is the probability of making a TYPE I error. Now consider what happens when we do three t
tests. There is .05 probability of making a TYPE I error on the first test, .05 probability of the
same error on the second test, and .05 probability on the third test. What happens is that these
errors are essentially additive, in that the chances of at least one TYPE I error among the three
tests much greater than .05. It is like the increased probability of drawing an ace from a deck of
cards when we can make multiple draws.
ANOVA allows us do an "overall" test of multiple groups to determine if there are any differences
among groups within the set. Notice that ANOVA does not tell us which groups among the three
groups are different from each other. The primary test in ANOVA is only to determine if there is
a significant difference among the groups somewhere.
You will study the theory and logic of analysis of variance (ANOVA). Recall that a t-test requires
a predictor variable that is dichotomous. The advantage of ANOVA over a t-test is that the
categorical predictor variable includes 3+ values (groups). Just like a t-test, the outcome
variable in ANOVA is quantitative and requires the calculation of group means.
In ANOVA, there are two levels of hypotheses. There is first the overall question of whether all
the group means are equal, or if there are some differences among the means somewhere. This
is called the omnibus null hypothesis test. The test is designed to show that the probability that
the group means are all equal is very low, leading to the researcher being able to legitimately
claim there are differences. This is done with the F test. In ANOVA, once the omnibus null
1
hypothesis is rejected, then one may legitimately use special tests, called post hoc tests, to
examine each of the pairs of groups in the set to determine which ones differ and which do not.
For instance, if an ANOVA is performed for three groups, the omnibus null hypothesis is that the
three groups have equal means. If that null is rejected, then the researcher may use special
post hoc tests to compare groups 1 & 2, groups 1 & 3, and groups 2 & 3. Each of these post
hoc tests are themselves null hypothesis tests, similar to the t tests which were studied
previously.
They are designed to control for multiple comparisons, or an inflation of the Type I error rate that
is a result of doing many tests with some fixed probability of error on each test. Most are based
on the assumption that the omnibus null has been rejected.
The Logic of a One-Way ANOVA
The ANOVA, or F-test, relies on predictor variables referred to as factors. A factor is a
categorical (nominal) predictor variable. The term "one-way" is applied to an ANOVA with only
one factor that is defined by two or more mutually exclusive groups. Technically, an ANOVA can
be calculated with only two groups, but the t-test is usually used instead. The one-way ANOVA
is usually calculated with three or more groups, which are often referred to as levels of the
factor.
If the ANOVA includes multiple factors, it is referred to as a factorial ANOVA. An ANOVA with
two factors is referred to as a "two-way" ANOVA; an ANOVA with three factors is referred to as
a "three-way" ANOVA, and so on. Factorial ANOVA is studied in Advanced Inferential Statistics.
In this course, we will focus on the theory and logic of the one-way ANOVA.
ANOVA is one of the most popular statistics used in psychological research. In nonexperimental
designs, the one-way ANOVA compares group means across naturally existing characteristics
of groups, such as political affiliation. In experimental designs, the one-way ANOVA compares
group means for participants randomly assigned to treatment conditions (for example, high
caffeine dose; low caffeine dose; control group).
Avoiding Inflated Type I Error
You may wonder why a one-way ANOVA is necessary. For example, if a factor has four groups
( k = 4), why not just run independent sample t tests for all pairwise comparisons (for example,
Group A versus Group B, Group A versus Group C, Group B versus Group C, et cetera)?
Warner (2013) points out that a factor with four groups involves six pairwise comparisons. The
issue is that conducting multiple pairwise comparisons with the same data leads to inflated risk
of a Type I error (incorrectly rejecting a true null hypothesis—getting a false positive). The
ANOVA protects the researcher from inflated Type I error by calculating a single omnibus test
that assumes all k population means are equal.
Although the advantage of the omnibus test is that it helps protect researchers from inflated
Type I error, the limitation is that a significant omnibus test does not specify exactly which group
means differ, just that there is a difference "somewhere" among the group means. A researcher
therefore relies on either (a) planned contrasts of specific pair wise comparisons determined
prior to running the F-test, or, (b) follow-up tests of pair wise comparisons, also referred to as
post-hoc tests, to determine exactly which pair wise comparisons are significant. Usually, if
planned contrasts are designed correctly, there is no need to perform the omnibus null test, and
the overall ANOVA is not necessary.
2
Hypothesis Testing in One-Way ANOVA
The null hypothesis of the omnibus test is that all k population means are equal, or H0: µ1 = µ2
=…µk. By contrast, the alternative hypothesis is usually articulated by stipulating that H0 is not
true. Keep in mind that this prediction does not imply that all groups must significantly differ from
one another on the outcome variable. In fact, for reasons beyond the scope of our present
discussion, it is not even strictly necessary that any two groups differ even if the omnibus null is
rejected.
Assumptions of One-Way ANOVA
The assumptions of ANOVA reflect assumptions of the t-test. ANOVA assumes independence
of observations. ANOVA assumes that outcome variable Y is normally distributed. ANOVA
assumes that the variance of Y scores is equal across all levels (groups) of the factor. These
ANOVA assumptions are checked in the same process used to check assumptions for the t-test
discussed earlier in the course—using the Shapiro-Wilk test and the Levene test.
Effect Size for a One-Way ANOVA
The effect size for a one-way ANOVA is eta squared (η2). It represents the amount of variance
in Y that is attributable to group differences. Recall the concept of sum of squares ( SS). Eta
squared for the one-way ANOVA is calculated by dividing the sum of squares of between-group
differences (SS-between) by the total sums of squares in the model (SS-total), which is reported
in SPSS output for the F-test. Eta squared for the one-way ANOVA is interpreted with <.02 as
"small," < .06 as "medium," and > .06 as "large."
References
Lane, D. M. (2013). HyperStat online statistics textbook. Retrieved from
http://davidmlane.com/hyperstat/index.html
Warner, R. M. (2013). Applied statistics: From bivariate through multivariate techniques (2nd
ed.). Thousand Oaks, CA: Sage Publications.
3