Download Definitions

Effect Size Definitions
Meta-analysis weights
Meta-analysis takes an average
Unit weights (unweighted average; w=1) (Bonett)
Sample size weights (w = N) (Schmidt & Hunter)
Inverse variance weights (w = 1/V) (Hedges & Olkin)
There are arguments in favor of each. We will mostly
focus on inverse variance weights.
Single Variable Effect Sizes
Use for central tendency
E.g., what is the graduation rate from college?
What is the time to complete college?
What is the proportion of female college graduates?
Proportion (Direct)
k
ESp  p 
N
ES = Effect size. P is the proportion of things of interest.
e.g., p = proportion field goals made from less than 40 yards.
Precision:
p(1  p)
Vp 
N
Proportion (Logit)
3
Logit (p)
2
1
0
-1
-2
-3
0.1
0.3
0.5
0.7
Proportion (p)
0.9
 p 
ESl  log e 

1  p 
Logit has nice statistical properties.
Precision VESi
1

np(1  p)
Aritmethic Mean
ESm
X

X
N
e.g., mean achievement test score.
S2
Precision V X 
N
Conventional Effect Sizes
Most effect sizes show the relations between two variables,
either a difference between groups (IV) on some criterion of
interest (DV), such as d, the standardized mean difference,
or an association between two continuous variables (e.g., the
correlation), or between two categorical variables (e.g. odds
ratios).
Mean Difference
(Unstandardized)
  1  2
D  X1  X 2
VD 
n1  n2 2
S pooled
n1n2
Used ONLY if measures are the same across all studies (e.g.,
used the Beck Depression Inventory to study the
effectiveness of a treatment for depression (experimental vs.
control group design).
S pooled
(n1  1) S12  (n2  1) S 22

n1  n2  2
Mean Difference
(Standardized)
1   2


S pooled
(n1  1) S12  (n2  1) S 22

n1  n2  2
X1  X 2
d
S pooled
n1  n2
d2
Vd 

n1n2
2(n1  n2 )
Spooled is the pooled Standard deviation. Note that the variance of
d depends upon the magnitude of d (actually delta, estimated by d).
The estimated standard deviation in Excel is stdev.s. Example:
=stdev.s(a1:a10)
Denominators of d and t
S pooled
(n1  1) S12  (n2  1) S 22

n1  n2  2
X1  X 2
d
S pooled
This is the pooled standard deviation –within group SD, the
yardstick for computing d.
S X1  X 2
(n1  1) s12  (n2  1) s22  1 1 
  

(n1  1)  (n2  1)  n1 n2 
t X1  X 2
X1  X 2

S X1  X 2
This is the standard error of the difference between means. This is
the yardstick for the independent samples t-test. Which will show
a larger difference between group means?
Mean Difference
(Standardized)
Bias correction:

3 
g  1 
d

 4df  1
Formulas from Borenstein et
al., 2009, p. 27
2

3 
Vg  1 
Vd

 4df  1
df  n1  n2  2
The effect size d is sometimes called ‘Cohen’s d’ and
the effect size g is sometimes called ‘Hedges’ g’ but
in practice they are essentially the same. It is now
conventional to use g.
Binary IV & DV – risk ratio
Events
Non-Events
Treated
A
B
n1
Control
C
D
n2
Total
Heart Attack
No attack
Treated
5
45
50
Control
10
40
50
100
RiskRatio 
A / n1
C / n2
LogRiskRat io  ln( RiskRatio )
VLogRiskRatio
1 1 1 1
   
A n1 C n2
Binary - odds ratio
Events
Non-Events
Treated
A
B
n1
Control
C
D
n2
Total
odds ratio 
A / B AD

C / D BC
 AD 
LogOddsRat io  log e 

BC


VLogOddsRatio
1 1 1 1
   
A B C D
Correlation (Pearson’s r)
Correlatio n  r
z z

r
x
(1  r 2 ) 2
Vr 
N 1
y
N
1  r 
z  .5 log e 
1  r 
1
Vz 
N 3
Fisher’s r to z transformation.
The Excel function for correlation is
correl(rangeX, rangeY). Example:
=correl(a1:a10, b1:10).
The r to z in Excel is
=atanh(correlation) e.g., =atanh(c11).
Class Exercise 1a
Group 1
Group 2
4
5
5
5
6
7
7
8
5
4
8
9
7
8
9
11
Compute Cohen’s d for these data. Compute Hedges’ g for
these data. I would use Excel if I were you.
Class Exercise 1b
Variable X
Variable Y
4
5
5
5
6
7
7
8
5
4
8
9
7
8
9
11
Compute the correlation coefficient r for these data (note the
data are the same as exercise 1a, but we have only one group of
people and two variables. Compute Fisher’s z for these data.