Download 2 - UW Canvas

Psych 200
Methods & Analysis
Dr. Kulstad for Dr. Lindgren
Fall 2008
Outline
1. Turn in homework
2. Stats in the news
3. ANOVA con’t
4. Bring to next class (Tuesday)
-- COMPLETED ungraded pop quiz (available on
Blackboard) as of end of class.
-- List of statistical tests for group project
3. Components of the F-statistic
Example
A researcher is interested in testing the
effectiveness of medication on reducing pain.
She is interested in examining the effectiveness
of Aspirin, Tylenol, and a placebo. She recruits
subjects and randomly assigns them to one of
the three groups. Participants perceptions on
how much their pain is reduced is measured.
What kind of design is this?
What is the IV? DV?
How many levels or conditions of the IV are
there?
What are the null & alternative hypotheses?
Xbar
Aspirin
Tylenol Placebo
3
2
2
5
2
1
3
4
3
5
4
2
4
3
2
3. Components of the F-statistic
•
Aspirin
MSwn: Basic formula
–
Add the squared deviations from the
3
mean for each condition & divide by df
5
Aspirin: (3-4)2 + (5-4)2+ (3-4)2 + (5-4)2 = 4
3
Tylenol: (2-3)2 + (4-3)2+ (2-3)2 + (4-3)2 = 4
5
Placebo: (2-2)2 + (1-2)2+ (3-2)2 + (2-2)2 = 2
–
Sum of all of the squared deviations in
Xbar 4
each condition: 4 + 4 + 2 = 10
–
df: N –k = 12 - 3 = 9
Sum of Squares
–
SSwn = 10 = 1.111
within (SSwn)
df
9
Mean Squares
within (MSwn)
Tylenol Placebo
2
2
2
1
4
3
4
2
3
2
df within
(dfwn)
3. Components of the F-statistic
•
Components of ANOVAs are often arrayed in tables
–
Let’s fill in what we know so far with our sample data…
Source
Sum of
Squares
df
Mean
square
10
9
1.111
Between
Within
Total
F
3. Components of the F-statistic
•
Mean square between groups (MSbn)
–
Estimate of variability of scores that occurs between
levels/conditions in a factor
•
How much does each level mean deviate from overall mean of
the experiment
Way to measure how much the levels means differ from one
another
Takes into account error AND effect of treatment
•
•
–
–
If null is true, MSbn is only estimating one population -- only
estimating σ2error like MSwn
If null is not true, MSbn is estimating error variance and treatment
variance
3. Components of the F-statistic
•
Mean square between groups (MSbn)
–
Basic formula
•
Σn(Xbar - GM)2
–
–
–
•
Calculate the deviation of a condition mean from the grand mean
& multiply by the number in that condition
Repeat for each condition
Add those values
Divide by df: k-1 (k = # conditions)
Grand mean (GM) =
mean of all scores in
the entire
experiment
3. Components of the F-statistic
Aspirin
•
Tylenol Placebo
MSbw: Basic formula
–
Calculate the deviation of a condition mean
3
2
from the grand mean (mean of ALL scores)
5
2
& multiply by the number in that condition
3
4
–
Repeat for each condition
5
4
–
Add those values
Xbar 4
3
Aspirin: 4* (4-3)2 = 4
Tylenol: 4 * (3-3)2 = 0
GM (Grand mean)
Placebo: 4 * (2-3)2 = 4
–
Sum for all conditions: 4 + 0 + 4 = 8
Sum of Squares
–
df: k-1 = 3 -1 = 2
between (SSbt)
–
SSbt = 8 = 4
df
2
Mean Squares
between (MSbt)
df between
(dfwn)
2
1
3
2
2
3
3. Components of the F-statistic
•
Back to our ANOVA table
Source
Sum of
Squares
df
Mean
square
Between
8
2
4
Within
10
9
1.111
F
Sum of Squares
total (SStot) =
SSbt + SSwn =
Σ(X- GM)2
df total (dftot) =
dfbt + dfwn =
N-1
Total
–
Now, we can complete it
Fobt = MSbt /MSwn
Source
Sum of
Squares
df
Mean
square
F
Between
8
2
4
3.60
Within
10
9
1.111
Total
18
11
3. Components of the F-statistic
•
Logic of the F ratio (Fobt)
–
To conduct an ANOVA, we compare MSbn / MSwn
•
•
If H0 is true
–
MSbn should = MSwn
–
NO treatment + Error = Error
–
Fobt = 1
If H0 is false
–
MSbn should be > MSwn
–
Treatment + Error vs. Error
–
Fobt > 1
–
Does not mean we reject H0 every time Fobt > 1
»
Effect has to be strong enough to rule out differences
due to chance alone
3. Components of the F-statistic
•
F-Distribution
–
Sample distribution that shows values of F that occur
when H0 is true
Positively skewed
Like t-distribution, F is a group of curves
–
–
•
–
Dependent on df
Unlike t-distribution, F, depends on 2 dfs
•
•
–
dfbw
dfwn
Need both to determine Fcrit
•
•
(By hand) use chart in back of book
(SPSS) if Fobt < α, reject H0,
3. Components of the F-statistic
•
Back to our data
–
–
–
–
Source
Sum of
Squares
df
Mean
square
F
Between
8
2
4
3.60
9
1.111
α = .05
Within
10
Total
18
Fcrit = 4.26
The exact p-value for Fobt = .071
What do we decide?
•
–
Fail to reject H0
How do we write that up?
•
Fobt > 1 &11not rejecting H0?Why?
Due to sample size (which
determines df), can’t rule out that
differences observed are simply
due to chance – i.e., treatment
effects aren’t “extreme”enough to
rule out chance.
A one-way analysis of variance was conducted to examine the
effect of three drugs on pain relief. There were no significant
differences in pain relief, F(2,9) = 3.60, p > .05. Therefore, there
was no evidence that Tylenol or Aspirin worked better than a
placebo.
3. Components of the F-statistic
•
What if data and results were different?
Xbar
–
–
–
–
–
Aspirin
Tylenol Placebo
3
2
2
5
2
3
Source
Sum of
Squares
df
Mean
square
F
1
Between
10.167
2
5.083
5.229
4
2
Within
8.750
9
.972
5
4
2
Total
18
11
4
3
1.75
α = .05
Fcrit = 4.26
The exact p-value for Fobt = .031
What do we decide?
What trouble do we run into?
4. Performing Post hoc Comparisons
•
Procedures to identify significant differences in
condition/level/group means
–
–
–
Variety of procedures
Some are more/less conservative
Book covers 2
•
Fisher’s protected t-test
–
•
Used if groups have unequal n’s
Tukey’s HSD multiple comparisons test
–
Used if groups have equal n’s
4. Performing Post hoc Comparisons
•
Fisher’s protected t-test
–
tobt =
(Xbar1 – Xbar2)
[ MSwn (1/n1 + 1/n2)]1/2
Same as independent samples t-test BUT use MSwn in
lieu of s2pool
Use this formula for every pair of means in experiment
Use t table in book & find tcrit (use two-tailed values)
–
–
–
•
–
df is dfwn
Remember not used for equal n’s
4. Performing Post hoc Comparisons
•
Tukey’s HSD (Honestly Significant Difference)
–
–
Used with equal n’s
Computes the minimum difference between means that
is required for them to differ significantly
Steps
–
•
Identify qk (table in book)
–
•
•
•
Need k (α, # levels, and dfwn)
Multiply qk by (MSwn/n)½
Determine the differences between all the means
Compare each difference to the HSD difference
–
–
If difference is > than HSD, means differ significantly
If difference is < HSD, means do not differ significantly
4. Performing Post hoc Comparisons
•
Which post hoc test would we run?
Xbar
–
–
–
–
Aspirin
Tylenol Placebo
3
2
2
5
2
3
Source
Sum of
Squares
df
Mean
square
F
1
Between
10.167
2
5.083
5.229
4
2
Within
8.750
9
.972
5
4
2
Total
18
11
4
3
1.75
α = .05
Determine which means differ significantly & write up
results using APA style & GST
qk =3.95
Aspirin & Placebo differ significantly
Tukey’s HSD (SPSS)
Multiple Comparisons
pain
Tukey HSD
95% Confidence Interval
Mean Difference
(I-J)
(I) drug
(J) drug
1
2
1.00000
.69722
.365
-.9466
2.9466
3
2.25000*
.69722
.025
.3034
4.1966
1
-1.00000
.69722
.365
-2.9466
.9466
3
1.25000
.69722
.226
-.6966
3.1966
1
-2.25000*
.69722
.025
-4.1966
-.3034
2
-1.25000
.69722
.226
-3.1966
.6966
2
3
Std. Error
*. The mean difference is significant at the 0.05 level.
Sig.
Lower Bound
Upper Bound
5. Summary of Steps
•
Initial steps
–
•
Computations
–
–
–
•
•
•
Determine null & alternative hypotheses, choose alpha,
check assumptions, collect data
Sums of squares
dfs
Mean squares
Identify Fcrit
If you reject the null, perform the appropriate post
hoc tests
Interpret & write up your results
6. Describing the relationship
•
Can also calculate confidence intervals
–
–
–
•
Upper bound: [( MSwn /n)1/2 * +tcrit]+ Xbar
Lower bound: [( MSwn /n)1/2 * -tcrit]+ Xbar
Calculated for every significant condition/mean/group
Can also calculate the effect size (eta squared or η2)
–
–
New correlation coefficient
Akin to a squared correlation coefficient
•
–
Remember squared correlation coefficient tells you proportion
of variance of accounted for
Formula:
SSbn / SStot
7. Power
•
•
F statistic (Fobt) : MSbn / MSwn
Maximize power through
–
Good design
•
Strong manipulation
–
•
Added control (lessen variability and error)
–
•
Decreases denominator
Larger n’s
–
–
–
–
Increase numerator
Increases dfwn
Minimizes MSwn
Decreases size of Fcrit
All of the above increase Fobt