Download Introduction to Meta

Introduction to Meta-Analysis
by Michael Borenstein, Larry V. Hedges , Julian P. T Higgins, and
Hannah R. Rothstein
PART 2 Effect Size and Precision
Summary of
Chapter 3: Overview
Chapter 4: Effect Sizes Based on Means
Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables)
Chapter 6: Effect Sizes Based on Correlations
Sarah McCann
CAMARADES: Bringing evidence to translational medicine
Chapter 3: Overview
Treatment Effects and Effect Sizes
Terminology:
Effect size – relationship between two variables e.g.
difference between males and females
Treatment effect – only appropriate when quantifying the
impact of a deliberate intervention e.g. the difference
between treated and control groups
Single group summary – estimating the mean or risk or
rate of a single population e.g. prevalence of a disease in
a particular region or mean scores on a test
CAMARADES: Bringing evidence to translational medicine
Chapter 3: Overview
How to Choose an Effect Size Index
1. The effect sizes from the different studies should be
comparable i.e. they measure (at least approximately)
the same thing.
2. Estimates of the effect size should be computable from
the information in the published reports (should not
require re-analysis of raw data, unless they’re known to
be available).
3. Need to be able to compute variance and confidence
intervals.
CAMARADES: Bringing evidence to translational medicine
Chapter 3: Overview
Summary Data Reported by
Primary Studies
Appropriate Effect Size
Means and standard deviations
(Chapter 4)
Raw difference in means
Standardised difference in means
Response ratio
Binary data e.g. events vs. non-events
(Chapter 5)
Risk ratio
Odds ratio
Risk difference
Correlation between two variables
(Chapter 6)
Correlation coefficient
CAMARADES: Bringing evidence to translational medicine
Chapter 4: Effect Sizes Based on Means
RAW (UNSTANDARDIZED) MEAN DIFFERENCE D
The outcome is reported on a meaningful scale and all
studies in the analysis use the same scale.
Meta-analysis is performed directly on the raw difference in
means.
CAMARADES: Bringing evidence to translational medicine
Chapter 4: Effect Sizes Based on Means
Let µ1 and µ2 be the true (population) means of two groups.
The population mean difference is defined as
We can compute an estimate, D, of the population mean
using the sample means from studies that use two
independent groups, paired groups or matched designs.
CAMARADES: Bringing evidence to translational medicine
Chapter 4: Effect Sizes Based on Means
Computing D from studies that use independent groups
The variance is calculated using the sample standard
deviations (S) and sample sizes (n) of the two groups.
CAMARADES: Bringing evidence to translational medicine
Chapter 4: Effect Sizes Based on Means
Computing D from studies that use matched groups or
pre-post scores
Pairs of participants are matched in some way (e.g. siblings
or patients at the same stage of disease) or the same
subject acts as their own control (e.g. scores pretreatment are compared to scores pot-treatment)
CAMARADES: Bringing evidence to translational medicine
Chapter 4: Effect Sizes Based on Means
STANDARDIZED MEAN DIFFERENCE, d and g
The scale of measurement differs across studies and it is
not meaningful to combine raw mean differences.
The mean difference in each study is divided by that study’s
standard deviation to create an index that is comparable
across studies.
CAMARADES: Bringing evidence to translational medicine
Chapter 4: Effect Sizes Based on Means
Let µ1 and σ1 and µ2 and σ2 be the true (population) means
and standard deviations of two groups. The population
mean difference is defined (assuming σ1 = σ2) as:
We can compute an estimate, d, of the population mean
using the sample means from studies that use two
independent groups, paired groups or matched designs.
CAMARADES: Bringing evidence to translational medicine
Chapter 4: Effect Sizes Based on Means
Computing d and g from studies that use independent
groups
By pooling the two estimates of the standard deviation, we
obtain a more accurate estimate of their common value.
CAMARADES: Bringing evidence to translational medicine
Chapter 4: Effect Sizes Based on Means
d has a slight bias, tending to overestimate the absolute
value of δ in small samples.
This bias can be removed with a simple correction, resulting
in an unbiased estimate sometimes called Hedges’ g.
To convert from d to Hedges’ g, a correction factor called J
is used:
And then:
CAMARADES: Bringing evidence to translational medicine
Chapter 4: Effect Sizes Based on Means
Computing d and g from studies that use pre-post scores
or matched groups
CAMARADES: Bringing evidence to translational medicine
Chapter 4: Effect Sizes Based on Means
Including different study designs in the same analysis
A systematic review may include some studies that use
independent groups and others that used matched
groups.
The effect size from each study can be computed using the
appropriate formula and all the studies can be combined
in the same analysis.
Direction of effect
is arbitrary, the
researcher must decide on a convention and apply it to
all studies, regardless of design.
CAMARADES: Bringing evidence to translational medicine
Chapter 4: Effect Sizes Based on Means
RESPONSE RATIOS
When the outcome is measured on a physical scale (e.g.
length, area, mass) and is unlikely to be zero, the ratio of
the two means can be used as the effect size.
Computations are carried out on a log scale.
CAMARADES: Bringing evidence to translational medicine
Chapter 4: Effect Sizes Based on Means
CAMARADES: Bringing evidence to translational medicine
Chapter 4: Effect Sizes Based on Means
The response ratio is computed as:
The log response ratio is computed as:
CAMARADES: Bringing evidence to translational medicine
Chapter 4: Effect Sizes Based on Means
SUMMARY POINTS
The raw mean difference (D) may be used as the effect size
when the outcome scale is either inherently meaningful or
well known due to widespread use. This effect size can only
be used when all studies in the analysis used precisely the
same scale.
The standardized mean difference (d or g) transforms all effect
sizes to a common metric, and thus enables us to include
different outcome measures in the same synthesis.
CAMARADES: Bringing evidence to translational medicine
Chapter 4: Effect Sizes Based on Means
SUMMARY POINTS
The response ratio (R) is often used in ecology. This effect size is
only meaningful when the outcome has a natural zero point,
but when this condition holds, it provides a unique
perspective on the effect size.
It is possible to compute an effect size and variance from studies
that used two independent groups, from studies that used
matched groups (or pre-post designs) and from studies that
used clustered groups. These effect sizes may then be
included in the same meta-analysis.
CAMARADES: Bringing evidence to translational medicine
Chapter 5: Effect Sizes Based on
Binary Data (2 x 2 Tables)
CAMARADES: Bringing evidence to translational medicine
Chapter 5: Effect Sizes Based on
Binary Data (2 x 2 Tables)
RISK RATIO
The ratio of two risks.
Risk of death in treated group is 5/100 and the risk of death in
the control group is 10/100, so the ratio of the two risks is
0.50.
CAMARADES: Bringing evidence to translational medicine
Chapter 5: Effect Sizes Based on
Binary Data (2 x 2 Tables)
CAMARADES: Bringing evidence to translational medicine
Chapter 5: Effect Sizes Based on
Binary Data (2 x 2 Tables)
ODDS RATIO
The ratio of two odds.
Odds of death in the treated group would be 5/95 or 0.0526 and
the odds of death in the control group would be 10/90 or
0.1111, so the ratio of the two odds would be 0.0526/0.1111
or 0.4737.
CAMARADES: Bringing evidence to translational medicine
Chapter 5: Effect Sizes Based on
Binary Data (2 x 2 Tables)
Why use log transformations?
The log transformation is needed to maintain symmetry in the
analysis.
Assume that one study reports that the risk is twice as high in Group A
while another reports that it is twice as high in Group B. Assuming
equal weights, these studies should balance each other, with a
combined effect showing equal risks (a risk ratio of 1.0).
However, on the ratio scale these correspond to risk ratios of 0.50 and
2.00, which would yield a mean of 1.25. By working with log values we
can avoid this problem. In log units the two estimates are +0.693 and
-0.693, which yield a mean of 0.00.
We convert this back to a risk ratio of 1.00, which is the correct value
for this data.
CAMARADES: Bringing evidence to translational medicine
Chapter 5: Effect Sizes Based on
Binary Data (2 x 2 Tables)
RISK DIFFERENCE
The difference between two risks.
Risk in the treated group is 0.05 and the risk in the control group
is 0.10, so the risk difference is -0.05.
No log transformation.
CAMARADES: Bringing evidence to translational medicine
Chapter 5: Effect Sizes Based on
Binary Data (2 x 2 Tables)
SUMMARY POINTS
We can compute the risk of an event (such as the risk of death)
in each group (for example, treated versus control). The ratio
of these risks then serves as an effect size (the risk ratio). The
difference in these risks can also serve as an effect size (the
risk difference).
We can compute the odds of an event (such as ratio of dying to
living) in each group (for example, treated versus control).
The ratio of these odds then serves as the odds ratio.
CAMARADES: Bringing evidence to translational medicine
Chapter 5: Effect Sizes Based on
Binary Data (2 x 2 Tables)
SUMMARY POINTS
To work with the risk ratio or odds ratio we transform all values
to log values, perform the analyses, and then convert the
results back to ratio values for presentation. To work with the
risk difference we work with the raw values.
When choosing an effect size index, consider:
The risk and odds ratios are relative measures – relatively
insensitive to differences in baseline events.
The risk difference is an absolute measure – very sensitive to
the baseline risk.
CAMARADES: Bringing evidence to translational medicine
Chapter 6: Effect Sizes Based on
Correlations
Computing r
Studies report a correlation between two continuous
variables.
The correlation coefficient can serve as the effect size
index.
Standardised to take into account the different metrics in
original scales.
CAMARADES: Bringing evidence to translational medicine
Chapter 6: Effect Sizes Based on
Correlations
CAMARADES: Bringing evidence to translational medicine