Download Hypothesis Testing * The 7-Step Procedure

R Programming
Data Analysis Module: One
Way Analysis of Variance
(ANOVA)
Data Analysis Module
 Basic Descriptive Statistics and Confidence Intervals
 Basic Visualizations
 Histograms
 Pie Charts
 Bar Charts
 Scatterplots
 Ttests/Bivariate testing
 One Sample
 Paired
 Independent Two Sample
 ANOVA
 Chi Square and Odds
 Regression Basics
2
Data Analysis Module: ANOVA
In a previous lecture, we covered several different forms of
t-tests.
For example, we used ttests to answer questions such as:
Are cars going over the speed limit on this residential street?
Did students who took a preparatory course, score better on
the standardized exam than students who did not take the
course?
Data Analysis Module: ANOVA
The general setting for all of these questions is that some
quantitative variable (speed, test scores) has been
measured for one or two categories of subjects.
What if we have more than two categories across which we
want to compare the value of some quantitative variable?
For example, lets say that we wanted to compare the mean
weight loss of subjects who were put on one of four diet
plans. For ease of discussion, lets call these plans A, B, C
and D.
Data Analysis Module: ANOVA
The following approach would be tempting…
H0: Plan A = Plan B
H0: Plan B = Plan C
H1: plan A  Plan B
H1: plan B  Plan C
H0: Plan C = Plan D
H0: Plan A = Plan C
H1: plan C  Plan D
H1: plan A  Plan C
H0: Plan A = Plan D
H0: Plan B = Plan D
H1: plan A  Plan D
H1: plan B  Plan D
Data Analysis Module: ANOVA
…but wrong.
Apart from being very cumbersome, there is a critical
problem – we are inflating our probability of making a type
1 error.
Think about that – lets use alpha = .05. If we ran 6 separate
tests, that would generate a cumulative probability of a
type 1 error of .3.
We could lower the alpha value to .05/6 – I hear you saying.
But this has its own problems – what happens if the number
of tests increase to 8 or 10? Our alpha value would become
so low, we would almost never reject the null (recall Power).
Data Analysis Module: ANOVA
What we need is a single test which will allow us to evaluate
all of the relationships simultaneously while using a
reasonable alpha level.
The test needs to be able to provide information regarding
differences in the mean values of multiple subject
groupings.
To accomplish this, we use the Analysis of Variance or
ANOVA procedure.
Data Analysis Module: ANOVA
Lets discuss how to use ANOVA to test a hypothesis by
returning to our dieters…
In this instance there are four levels (diet plans) to a single
factor (weight loss).
The hypothesis statements would look like this:
H0: All level means are equal. In other words, all four of the
diet plans generate approximately the same amount of
weight loss.
H1: Not all of the level means are equal. In other words, at
least one of the plans’ weight loss mean is statistically
significant different from the other plans’ means.
Data Analysis Module: ANOVA
Prior to executing the test, we must check for three
important assumptions about our data:
1. All the groups are normally distributed.
2. All the populations sampled have approximately equal
variance (you can check this by generating side-by-side
boxplots). The rule of thumb is that the largest std is <2x
the smallest std.
3. The samples of the groups are independent of each
other and subjects within the groups were randomly
selected.
As with most, but not all, statistical tests, if our samples are
large, we can relax our assumptions and work around non
normal data.
Data Analysis Module: ANOVA
Lets examine the hypothesis statements in more detail:
H0: µa = µb = µc = µd
H1: µa ≠ µb ≠ µc ≠ µd
Consider – what would the hypothesized distributions look
like under H0 and H1?
Data Analysis Module: ANOVA
Ok. We understand the concept, we have the hypotheses,
we have the assumptions – we need a test statistic.
In ANOVA, we use the F-distribution. In the science of
statistics, whenever you need to evaluate a ratio of
variances you will be using an F-statistic.
The ratio in question here is:
The variation BETWEEN the groups
The variation WITHIN the groups
Question – what kind of value would indicate difference
versus no difference?
Data Analysis Module: ANOVA
The result of this ratio is the F-statistic. As the number of
groups and observations increases, the distribution will start
to appear normal.
Lets start working with an example…
Data Analysis Module: ANOVA
Returning to the diet plans…
PLAN Mean
PLAN A
14
14
20
22
26
27
20.50
PLAN B
15
18
23
25
28
30
23.17
PLAN C
32
36
40
42
45
45
40.00
PLAN D
33
38
42
44
46
47
41.67
OVERALL MEAN
31.33
Data Analysis Module: ANOVA
Our hypotheses statements would be:
H0: The four diets plans have the same results (the mean
weight loss is the same)
H1: At least one of the diet plans has a different result (the
mean weight loss is different)
We will now calculate our test statistic:
The variation BETWEEN the groups
The variation WITHIN the groups
Data Analysis Module: ANOVA
To calculate the F-Statistic, we use the following table:
SOURCE
SUM OF
SQUARES
DEGREES
OF
FREEDOM
MEAN
SQUARE
F-stat
BETWEEN
SSB
# levels – 1
SSB/(# levels – 1)
{SSB/(# levels – 1)}
{SSW(n- # levels)}
WITHIN
SSW
n- # levels
SSW/(n- # levels)
TOTAL
SST
(SSB + SSW)
n-1
Data Analysis Module: ANOVA
For those who are interested:
SST
_
= SSW
_
+ SSB
_
ij(Xij-X)2 = ij(Xij-Xj)2 + nj(Xj-X)2
Data Analysis Module: ANOVA
For the present problem:
SOURCE
1
SUM OF
SQUARES
DEGREES
OF
FREEDOM
MEAN
SQUARE
BETWEEN1
2195.67
3
731.89
WITHIN2
601.67
20
30.08
TOTAL
2797.34
23
SSB = 6(10.832 + 8.172 + 8.672 +10.332)
2SSW
= (159.50 + 166.83 + 134 + 141.33) = 601.67
F-stat
24.33
Data Analysis Module: ANOVA
Now…what to do with an F-statistic of 24.33?
This is a fairly strong statistic – recall that as the variance ratio
approaches 1, the null is true. As the variance ratio grows larger
than 1, we can more confidently reject the null.
As with all test statistics, this result will translate into a p-value. The
p-value associated with this statistic is less than .001. Based upon
this result, we can confidently reject the null hypothesis and
conclude that at least one of the results is different.
Data Analysis Module: ANOVA
We are going to use some simple
ANOVA code:
a1 <- aov (y ~ x)
Where y is the quantitative continuous variable and x is
the categorical variable with more than 3 levels.
a1
summary(a1)
require(graphics)
summary(a1 <- aov((y ~ x))
TukeyHSD(a1, “x", ordered = TRUE)
plot(TukeyHSD(a1, “x"))