Download Jaehun on Power Analysis

Power Analysis and Meta-analysis
Jaehun Jung
10-27-201
H676
Objectives
• Discuss the factors that determine power and explore how
the value of these factors may change as we move from a
primary study to meta-analysis
• Review the process of power analysis for primary studies,
and then show how the same process can be extended for metaanalysis.
• Answer the question “how many studies do you need?”
and retrospective statistical power.
What it is and why it is important
• Power is the probability of detecting an effect, given that the
effect is really there.
• It is the probability of rejecting the null hypothesis when it is in
fact false.
• Is a good way of making sure that you have thought through
every aspect of your study and the statistical analysis before you
start conducting reviews.
When is it used?
• In primary studies
• Determine appropriate sample size
• In meta-analysis
• Already control over some variables (i.e., sample size and # of studies
meet the inclusion criteria)
• Use it when figure it out how many studies we need to include
(Valentine’s article)
Factors that Affect Power in Primary
• Expected effect size
• Sample size
• Alpha
• Power
Example
• Sample size = 25
• Effect size = .30
• Alpha = .05
• Power? 0.1840 => way low power!
Power Analysis in Meta-analysis
• Logic of power analysis for meta-analysis is similar to primary
• The precision reflect both sample size and # of studies
• Differences in precision whether it is fixed- or random-effect
• Throughout literature review, you might get sense of the factors
(i.e. effect size, sample size and # of studies that meet your criteria)
Factors Affecting Power in Meta-analysis
• Expected effect size
• Sample Size and # of Studies
• Alpha
• Fixed- or Random-effects
• Why does it matter?
Power Analysis in Fixed Effect Model
• The same formula and process, but…..
• If all studies had the same variance, then Vm = Vy/k
• So, the Lambda will be different
Power Analysis in Random Effect Model
• The same formula and process, but…..
• Within- and between-studies variance
• Obtain within-studies variance using same procedures as for
fixed• Obtain between-studies variance through pilot study, but.....
• Hedges and Pigott propose a convention
Small = 1.33 * the within-study variance
Medium = 1.67 * the within-study variance
Large = 2 * the within-study variance
Example
• Sample size = 25
• Effect size = .30
• Alpha = .05
• K = 10
• Random-effect Model
Power for a test of homogeneity
• Asks whether or not the between-studies dispersion is more
than would be expected by chance
• Ratio of between- and within-studies variance
• Number of studies
• Alpha
Example (Random-effect Model)
• # of studies = 6
• Large amount of dispersion
• Alpha = .05
• Use function CHINV(alpha, df) for critical value
• Power=1 – CHIDIST (x, df)
• Power = 0.3541
Discussion Questions 1
• Is there anyone who conduct power analysis on your study?
• Is there anyone who want to share with your power analysis?
•
•
•
•
What is your expected effect size?
What is the average sample size?
How many research do you have?
What is your desired power in your study?
How many studies are needed?
• Power = 1 – NORMSDIST(1.64 - Lambda)
• 80% = 1 – NORMSDIST(-.842) => yield 0.2 * if it one-side
• Thus….. Lambda = 1.64 + 0.842 = 2.482
Retrospective Power Analysis
• Retrospective power analysis using observed values can not add
information to the analysis already done.
• However, retrospective power analysis using some other value
that is meaning in the context can add information regarding
results.
Confidence Interval Estimation
• Lower bound of the CI tells us whether the result is statistically
significant and minimum likely size of the effect
• Upper bound of the CI gives us to assess the “best case” for the
intervention’s impact.
• The Width of the CI provides the full range of plausible
parameters that are consistent with the results of the mataanalysis
Discussion Question 2
• Should a synthesis be carried out if power is low?
Yes, it should! But why?