Download Hypothesis Testing

ANOVA
(Analysis of Variance)
Martina Litschmannová
[email protected]
K210
The basic ANOVA situation
 Two variables: 1 Categorical, 1 Quantitative
 Main Question: Do the (means of) the quantitative variables
depend on which group (given by categorical variable) the
individual is in?
 If categorical variable has only 2 values:
 null hypothesis: 𝜇1 − 𝜇2 = 𝐷
 ANOVA allows for 3 or more groups
An example ANOVA situation
 Subjects: 25 patients with blisters
 Treatments: Treatment A, Treatment B, Placebo
 Measurement: # of days until blisters heal
 Data [and means]:
A: 5,6,6,7,7,8,9,10
B: 7,7,8,9,9,10,10,11
P: 7,9,9,10,10,10,11,12,13
 Are these differences significant?
[7.25]
[8.875]
[10.11]
Informal Investigation
 Graphical investigation:
 side-by-side box plots
 multiple histograms
 Whether the differences between the groups are
significant depends on
 the difference in the means
 the standard deviations of each group
 the sample sizes
 ANOVA determines p-value from the F statistic
Side by Side Boxplots
13
12
11
days
10
9
8
7
6
5
A
B
treatment
P
What does ANOVA do?
At its simplest (there are extensions) ANOVA tests the following
hypotheses:
H0: The means of all the groups are equal.
HA: Not all the means are equal.
• Doesn’t say how or which ones differ.
• Can follow up with “multiple comparisons”.
Note: We usually refer to the sub-populations as “groups” when
doing ANOVA.
Assumptions of ANOVA
 each group is approximately normal
 check this by looking at histograms and/or normal
quantile plots, or use assumptions
 can handle some nonnormality, but not severe outliers
 test of normality
 standard deviations of each group are approximately
equal
 rule of thumb: ratio of largest to smallest sample st.
dev. must be less than 2:1
 test of homoscedasticity
Normality Check
We should check for normality using:
• assumptions about population
• histograms for each group
• normal quantile plot for each group
• test of normality (Shapiro-Wilk, Liliefors, AndersonDarling test, …)
With such small data sets, there really isn’t a really good way
to check normality from data, but we make the common
assumption that physical measurements of people tend to be
normally distributed.
Shapiro-Wilk test
 One of the strongest tests of normality. [Shapiro, Wilk]
Online computer applet (Simon Ditto, 2009) for this test can be
found here.
Standard Deviation Check
Variable
days
treatment
A
B
P
N
8
8
9
Mean
7.250
8.875
10.111
Median
7.000
9.000
10.000
StDev
1.669
1.458
1.764
Compare largest and smallest standard deviations:
 largest: 1,764
 smallest: 1,458
 1,764/1,458=1,210<2 OK
Note: Std. dev. ratio greather then 2 signs heteroscedasticity.
ANOVA Notation
1
2
…
k
𝑋11
𝑋12
…
𝑋1𝑘
⋮
⋮
⋮
𝑋𝑛1 1
𝑋𝑛22
𝑋𝑛𝑘 𝑘
Sample Size
𝑛1
𝑛2
…
𝑛𝑘
Sample average
𝑋1
𝑋2
…
𝑋𝑘
Sample Std. Deviation
𝑠1
𝑠2
…
𝑠𝑘
Sample
Group
Number of Individuals all together :
Sample means:
Grand mean:
Sample Standard Deviations:
𝑘
𝑖=1 𝑛𝑖 ,
1 𝑛𝑖
𝑋 ,
𝑛𝑖 𝑗=1 𝑖𝑗
1 𝑘
𝑛𝑖
𝑋=
𝑋 ,
𝑛 𝑖=1 𝑗=1 𝑖𝑗
1
𝑛𝑖
𝑠𝑖2 =
𝑋 −
𝑛𝑖 −1 𝑗=1 𝑖𝑗
𝑛=
𝑋𝑖 =
𝑋𝑖
2
.
Levene Test
Null and alternative hypothesis:
H0: 𝜎12 = 𝜎22 = ⋯ = 𝜎𝑘2 , HA: 𝐻0
Test Statistic:
𝐹𝑍 =
where 𝑍𝑖𝑗 = 𝑋𝑖𝑗 − 𝑋𝑖 , 𝑍𝑖 =
𝑆𝑆𝑍𝐵 =
𝑘
𝑖=1 𝑛𝑖
2
𝑆𝑆𝑍𝐵
𝑘−1
𝑆𝑆𝑍𝑒
𝑛−𝑘
𝑛𝑖
𝑍
𝑗=1 𝑖𝑗
,
𝑛𝑖
𝑍𝑖 − 𝑍 , 𝑆𝑆𝑍𝑒 =
,𝑍 =
𝑘
𝑖=1
𝑛𝑖 𝑍𝑖𝑗
𝑗=1 𝑛 ,
2
𝑛𝑖
𝑗=1 𝑍𝑖𝑗 − 𝑍𝑖 .
𝑘
𝑖=1
p-value: 𝑝−𝑣𝑎𝑙𝑢𝑒 = 1 − 𝐹0 𝑥𝑂𝐵𝑆 , where 𝐹0 𝑥 is CDF of
Fisher-Snedecor distribution with 𝑘 − 1, 𝑛 − 𝑘 degrees of
freedom.
How ANOVA works (outline)
ANOVA measures two sources of variation in the data and
compares their relative sizes.
How ANOVA works (outline)
 Sum of Squares between Groups,
2
𝑆𝑆𝐵 = 𝑘𝑖=1 𝑛𝑖 𝑋𝑖 − 𝑋 ,
resp. Mean of Squares – between groups
𝑀𝑆𝐵 =
𝑆𝑆𝐵
,
𝑘−1
Difference between
Means
where 𝑘 − 1 is degrees of freedom 𝑑𝑓𝐵 .
 Sum of Squares – errors
𝑆𝑆𝑒 =
𝑘
𝑖=1
𝑛𝑖
𝑗=1
𝑋𝑖𝑗 − 𝑋𝑖
2
=
resp. Mean of squares - error
𝑀𝑆𝑒 =
𝑆𝑆𝑒
,
𝑛−𝑘
where 𝑛 − 𝑘 is degrees of freedom 𝑑𝑓𝑒 .
𝑘
𝑖=1
𝑛𝑖 − 1 𝑠𝑖2 ,
Difference
within
Groups
The ANOVA F-statistic is a ratio of the Between Group
Variaton divided by the Within Group Variation:
𝑀𝑆𝐵
𝐹=
𝑀𝑆𝑒
A large F is evidence against H0, since it indicates that
there is more difference between groups than within
groups.
ANOVA Output
Source of Variation
Between Groups
Within Groups
Total
SS
34,74
59,26
94,00
DF
2
22
24
MS
17,37
2,69
F
6,45
p-value
0,006
How are these computations made?
Source of
Variation
Degrees of
Freedom
Sum of Squares
Mean of
Squares
𝑘
Between
Groups
Within
Groups
𝑆𝑆𝐵 =
𝑛𝑖 𝑋𝑖 − 𝑋
𝑆𝑆𝑒 =
𝑛𝑖 − 1 𝑠𝑖2
𝑖=1
𝑘 𝑛𝑖
𝑋𝑖𝑗 − 𝑋
𝑖=1 𝑗=1
2
𝑝−𝑣𝑎𝑙𝑢𝑒
1 − 𝐹0 𝑥𝑂𝐵𝑆
𝑑𝑓𝐵 = 𝑘 − 1
𝑀𝑆𝐵 =
𝑆𝑆𝐵
𝑑𝑓𝐵
𝑀𝑆𝐵
𝑀𝑆𝑒
𝑑𝑓𝑒 = 𝑛 − 𝑘
𝑀𝑆𝑒 =
𝑆𝑆𝑒
𝑑𝑓𝑒
---
---
𝑑𝑓𝑇 = 𝑛 − 1
---
---
---
𝑖=1
𝑘
𝑆𝑆𝑇 =
Total
2
𝐹
Using the results of exploratory analysis and ANOVA test, verify that the age
of a statistically significant effect on BMI.
Count
Average
Variance
-----------------------------------------------------------------------------<35 let
53
25,0796
10,3825
35 - 50 let
123
25,9492
16,2775
>50 let
76
26,0982
12,3393
------------------------------------------------------------------------------Total
252
25,8113
13,8971
58
BMI
48
38
28
18
méně než 35 let
od 35 do 50 let
více než 50 let
Using the results of exploratory analysis and ANOVA test, verify that the age
of a statistically significant effect on BMI.
Assumptions:
1. Normality
2. Homoskedasticita
H0: 𝜎12 = 𝜎22 = 𝜎32 , HA: 𝐻0 , 𝑝 − 𝑣𝑎𝑙𝑢𝑒 = 0,129 (Levene test)
Using the results of exploratory analysis and ANOVA test, verify that the age
of a statistically significant effect on BMI.
Null and alternative hypothesis:
H0: 𝜇1 = 𝜇2 = 𝜇3 , HA: 𝐻0
Calculating of p-value:
Count
Average
Variance
-----------------------------------------------------------------------------<35 let
53
25,0796
10,3825
35 - 50 let
123
25,9492
16,2775
>50 let
76
26,0982
12,3393
------------------------------------------------------------------------------Total
252
25,8113
13,8971
𝑆𝑆𝐵 =
𝑘
𝑖=1 𝑛𝑖
𝑆𝑆𝑒 =
𝑘
𝑖=1
𝑋𝑖 − 𝑋
2
= 53 ∙ 25,1 − 25,8 2 + 123 ∙ 25,9 − 25,8
+76 ∙ 26,1 − 25,8 2 =34,0
2
𝑛𝑖 − 1 𝑠𝑖2 = 52 ∙ 10,4 + 122 ∙ 16,3 + 75 ∙ 12,3 = 3451,9
+
Using the results of exploratory analysis and ANOVA test, verify that the age
of a statistically significant effect on BMI.
Null and alternative hypothesis:
H0: 𝜇1 = 𝜇2 = 𝜇3 , HA: 𝐻0
Calculating of p-value:
Source of
Variation
Degrees of
Freedom
Sum of Squares
Mean of
Squares
𝑘
Between
Groups
Within
Groups
𝑆𝑆𝐵 =
𝑛𝑖 𝑋𝑖 − 𝑋
𝑆𝑆𝑒 =
𝑛𝑖 − 1 𝑠𝑖2
𝑖=1
𝑘 𝑛𝑖
𝑋𝑖𝑗 − 𝑋
𝑖=1 𝑗=1
2
𝑝−𝑣𝑎𝑙𝑢𝑒
1 − 𝐹0 𝑥𝑂𝐵𝑆
𝑑𝑓𝐵 = 𝑘 − 1
𝑀𝑆𝐵 =
𝑆𝑆𝐵
𝑑𝑓𝐵
𝑀𝑆𝐵
𝑀𝑆𝑒
𝑑𝑓𝑒 = 𝑛 − 𝑘
𝑀𝑆𝑒 =
𝑆𝑆𝑒
𝑑𝑓𝑒
---
---
𝑑𝑓𝑇 = 𝑛 − 1
---
---
---
𝑖=1
𝑘
𝑆𝑆𝑇 =
Total
2
𝐹
Using the results of exploratory analysis and ANOVA test, verify that the age
of a statistically significant effect on BMI.
Null and alternative hypothesis:
H0: 𝜇1 = 𝜇2 = 𝜇3 , HA: 𝐻0
Calculating of p-value:
Source of
Variation
Between
Groups
Within
Groups
Sum of Squares
Degrees of
Freedom
34,0
𝑑𝑓𝐵 = 𝑘 − 1
3451,9
𝑑𝑓𝑒 = 𝑛 − 𝑘
𝑘
Total
𝑛𝑖
𝑆𝑆𝑇 =
𝑋𝑖𝑗 − 𝑋
𝑖=1 𝑗=1
2
𝑑𝑓𝑇 = 𝑛 − 1
Mean of
Squares
𝑆𝑆𝐵
𝑀𝑆𝐵 =
𝑑𝑓𝐵
𝑆𝑆𝑒
𝑀𝑆𝑒 =
𝑑𝑓𝑒
𝐹
𝑝−𝑣𝑎𝑙𝑢𝑒
𝑀𝑆𝐵
𝑀𝑆𝑒
1 − 𝐹0 𝑥𝑂𝐵𝑆
---
---
---
---
---
Using the results of exploratory analysis and ANOVA test, verify that the age
of a statistically significant effect on BMI.
Null and alternative hypothesis:
H0: 𝜇1 = 𝜇2 = 𝜇3 , HA: 𝐻0
Calculating of p-value:
Source of
Variation
Between
Groups
Within
Groups
Total
Sum of Squares
Degrees of
Freedom
34,0
𝑑𝑓𝐵 = 𝑘 − 1
3451,9
𝑑𝑓𝑒 = 𝑛 − 𝑘
3485,9
𝑑𝑓𝑇 = 𝑛 − 1
Mean of
Squares
𝑆𝑆𝐵
𝑀𝑆𝐵 =
𝑑𝑓𝐵
𝑆𝑆𝑒
𝑀𝑆𝑒 =
𝑑𝑓𝑒
---
𝐹
𝑝−𝑣𝑎𝑙𝑢𝑒
𝑀𝑆𝐵
𝑀𝑆𝑒
1 − 𝐹0 𝑥𝑂𝐵𝑆
---
---
---
---
Using the results of exploratory analysis and ANOVA test, verify that the age
of a statistically significant effect on BMI.
Null and alternative hypothesis:
H0: 𝜇1 = 𝜇2 = 𝜇3 , HA: 𝐻0
Calculating of p-value:
Source of
Variation
Between
Groups
Within
Groups
Total
Sum of Squares
Degrees of
Freedom
34,0
𝑑𝑓𝐵 = 𝑘 − 1
3451,9
𝑑𝑓𝑒 = 𝑛 − 𝑘
3485,9
𝑑𝑓𝑇 = 𝑛 − 1
k … number of sanmples 𝑘 = 3
n … total sample si𝑧𝑒 𝑛 = 252
Mean of
Squares
𝑆𝑆𝐵
𝑀𝑆𝐵 =
𝑑𝑓𝐵
𝑆𝑆𝑒
𝑀𝑆𝑒 =
𝑑𝑓𝑒
---
𝐹
𝑝−𝑣𝑎𝑙𝑢𝑒
𝑀𝑆𝐵
𝑀𝑆𝑒
1 − 𝐹0 𝑥𝑂𝐵𝑆
---
---
---
---
Using the results of exploratory analysis and ANOVA test, verify that the age
of a statistically significant effect on BMI.
Null and alternative hypothesis:
H0: 𝜇1 = 𝜇2 = 𝜇3 , HA: 𝐻0
Calculating of p-value:
Source of
Variation
Between
Groups
Within
Groups
Total
Degrees of
Freedom
Sum of Squares
34,0
3451,9
3485,9
/
/
2
249
251
=
=
Mean of
Squares
𝑆𝑆𝐵
𝑀𝑆𝐵 =
𝑑𝑓𝐵
𝑆𝑆𝑒
𝑀𝑆𝑒 =
𝑑𝑓𝑒
---
𝐹
𝑝−𝑣𝑎𝑙𝑢𝑒
𝑀𝑆𝐵
𝑀𝑆𝑒
1 − 𝐹0 𝑥𝑂𝐵𝑆
---
---
---
---
Using the results of exploratory analysis and ANOVA test, verify that the age
of a statistically significant effect on BMI.
Null and alternative hypothesis:
H0: 𝜇1 = 𝜇2 = 𝜇3 , HA: 𝐻0
Calculating of p-value:
Source of
Variation
Between
Groups
Within
Groups
Total
Sum of Squares
Degrees of
Freedom
Mean of
Squares
𝐹
𝑝−𝑣𝑎𝑙𝑢𝑒
34,0
2
17,0
𝑀𝑆𝐵
𝑀𝑆𝑒
1 − 𝐹0 𝑥𝑂𝐵𝑆
3451,9
249
13,9
---
---
3485,9
251
---
---
---
Using the results of exploratory analysis and ANOVA test, verify that the age
of a statistically significant effect on BMI.
Null and alternative hypothesis:
H0: 𝜇1 = 𝜇2 = 𝜇3 , HA: 𝐻0
Calculating of p-value:
Source of
Variation
Between
Groups
Within
Groups
Total
Sum of Squares
Degrees of
Freedom
Mean of
Squares
𝐹
𝑝−𝑣𝑎𝑙𝑢𝑒
34,0
2
17,0
1,23
0,294
3451,9
249
13,9
---
---
3485,9
251
---
---
---
𝑝 − 𝑣𝑎𝑙𝑢𝑒 = 1 − 𝐹 1,23 = 0,294 ,
where F(x) is CDF of Fisher-Snedecor distribution with 2 , 249 degrees of freedom
Using the results of exploratory analysis and ANOVA test, verify that the age
of a statistically significant effect on BMI.
Null and alternative hypothesis:
H0: 𝜇1 = 𝜇2 = 𝜇3 , HA: 𝐻0
Calculating of p-value:
Source of
Variation
Between
Groups
Within
Groups
Total
Sum of Squares
Degrees of
Freedom
Mean of
Squares
𝐹
𝑝−𝑣𝑎𝑙𝑢𝑒
34,0
2
17,0
1,23
0,294
3451,9
249
13,9
---
---
3485,9
251
---
---
---
Result:
We dont reject null hypothesis at the significance level 0,05. There is not a
statistically significant difference between the means of BMI depended on
the age.
Where’s the Difference?
Once ANOVA indicates that the groups do not all appear to have
the same means, what do we do?
Analysis of Variance for days
Source
DF
SS
MS
treatmen
2
34.74
17.37
Error
22
59.26
2.69
Total
24
94.00
Level
A
B
P
N
8
8
9
Pooled StDev =
Mean
7.250
8.875
10.111
1.641
StDev
1.669
1.458
1.764
F
6.45
P
0.006
Individual 95% CIs For Mean
Based on Pooled StDev
----------+---------+---------+-----(-------*-------)
(-------*-------)
(------*-------)
----------+---------+---------+-----7.5
9.0
10.5
Clearest difference: P is worse than A (CI’s don’t overlap)
Multiple Comparisons
Once ANOVA indicates that the groups do not all have the same
means, we can compare them two by two using the 2-sample t
test.
 We need to adjust our p-value threshold because we are doing
multiple tests with the same data.
 There are several methods for doing this.
 If we really just want to test the difference between one pair of
treatments, we should set the study up that way.
Bonferroni method – post hoc analysis
We reject null hypothesis if
𝑥𝐼 − 𝑥𝐽 ≥ 𝑡𝑛−𝑘 1
𝛼∗
−
2
𝑀𝑆𝑒
where 𝛼 ∗ is correct significance level, 𝛼 ∗ =
𝑡
𝛼∗
1− 2
𝑛 − 𝑘 is 1
𝛼∗
−
2
1
𝑛𝐼
+
𝛼
𝑘
2
,
1
,
𝑛𝐽
is quantile of Student distribution
with 𝑛 − 𝑘 degrees of freedom.
Kruskal-Wallis test
 The Kruskal–Wallis test is most commonly used when there
is one nominal variable and one measurement variable,
and the measurement variable does not meet the
normality assumption of an ANOVA.
 It is the non-parametric analogue of a one-way ANOVA.
Kruskal-Wallis test
 Like most non-parametric tests, it is performed on ranked
data, so the measurement observations are converted to
their ranks in the overall data set: the smallest value gets a
rank of 1, the next smallest gets a rank of 2, and so on. The
loss of information involved in substituting ranks for the
original values can make this a less powerful test than an
anova, so the anova should be used if the data meet the
assumptions.
 If the original data set actually consists of one nominal
variable and one ranked variable, you cannot do an anova
and must use the Kruskal–Wallis test.
1. The farm bred three breeds of rabbits. An attempt was made
(rabbits.xls), whose objective was to determine whether
there is statistically significant (conclusive) the difference in
weight between breeds of rabbits. Verify.
2. The effects of three drugs on blood clotting was determined.
Among other indicators was determined the thrombin time.
Information about the 45 monitored patients are recorded in
the file thrombin.xls. Does the magnitude of thrombin time
depend on the used preparation?
Study materials :
 http://homel.vsb.cz/~bri10/Teaching/Bris%20Prob%20&%20Stat.pdf
(p. 142 - p.154)
 Shapiro, S.S., Wilk, M.B. An Analysis of Variance Test for Normality
(Complete Samples). Biometrika. 1965, roč. 52, č. 3/4, s. 591-611.
Dostupné z: http://www.jstor.org/stable/2333709.