Download Are multiple primary outcomes analysed appropriately in RCTs?

Are multiple primary outcomes analysed
appropriately in RCTs?
Vicki Vickerstaff
Priment Clinical Trials Unit
Why use multiple outcomes?
• A single outcome may not be sufficient
• In complex conditions several outcomes may
need to be investigated to fully assess all aspects
of the disorder
• Example: relapse to drug use and frequency of
substance use in 90 days (Bowen et al, 2014)
Statistical methods to account for multiplicity
• Separate univariate analyses and adjust the p-values
(using Bonferroni, Holm, Hochberg etc.)
• Composite primary outcome
• Multivariate regression
Separate univariate analysis
with an adjustment to the p-values
 Easy to implement
P-value
P-value
x Conservative approach
when there are many
outcomes or when the
outcomes are correlated
Composite outcome
Combine multiple measurements into a
single variable
 Takes account of multiplicity without
the need to adjust the p-values
x It is not clear where the main effect is
P-value
x Misleading conclusions can be drawn
when a number of disparate outcome
are combined
Multivariate regression model
 Takes account of correlations
between outcomes
 Can either measure the
intervention effect for each
outcome or measure an
overall intervention effect
P-value P-value
If the intervention effect is
measured for each outcome then
an adjustment for the multiplicity
will be required
Review
• Review of randomised trials
• Between July 2011 and June 2014
• Vickerstaff, Victoria, et al. "Are multiple primary outcomes
analysed appropriately in randomised controlled trials? A
review." Contemporary clinical trials 45 (2015): 8-12.
Results
• 289 RCTs identified
– 209 trials included
– 80 phase 2 trial excluded
• 32% (67) reported > 1 primary outcomes
• 68% (142) reported 1 primary outcome
Results
> 1 primary
outcome reported
n= 67
Multiple primary
outcomes
n=60
Separate testing
n=58
Multiplicity
adjustment
n=13
32% of all RCTs
Co-primary
outcomes
n=7
Simultaneous
testing
n= 2
No multiplicity
adjustment
n=45
Adjustments:
Bonferroni (6)
Holm (2)
Hochberg-Benjamini (1)
Sidak (1)
Dunnett (1)
Sequential (2)
Recommendations for reporting
• Specify the primary and secondary outcomes, methods of
measurements and time points of interest
• Any aspects of multiplicity “should be identified in the
protocol; adjustment should always be considered and the
details of any adjusting procedure…should be set out in
the analysis plan” (ICH guidelines)
Current research
• Multivariate methods are infrequently used in the analysis
of clinical trials
• Comparing multivariate method to univariate method using
a simulation study
• Various outcomes types
– Continuous, binary, mixture of continuous and binary
• Various types of missingness
– Complete data, MCAR, MAR, MNAR
0.0
0.2
0.4
0.6
Correlation
0.8
1.0
0.00 0.05 0.10 0.15 0.20
FWER
0.8
0.7
0.6
Power
0.9
1.0
Simulations: Complete data
0.0
0.2
0.4
0.6
Correlation
0.8
1.0
0.8
0.7
0.6
Power
0.9
1.0
Simulations: MCAR 30%, 50%
0.0
0.2
0.4
0.6
Correlation
0.8
1.0
Results
• Multivariate methods may increase the power
when
– the outcomes are correlated (𝜌 > 0.4) and
– there are missing data
• When using 2 outcomes, the Bonferroni
adjustment keeps the FWER at the desired level
References
•
Roffman, J., et al. "Randomized multicenter investigation of folate plus vitamin
B12 supplementation in schizophrenia." JAMA psychiatry 70.5 (2013): 481489.
•
Goldstein H. Multilevel statistical models. Vol 922: Wiley. com; 2011.
•
Vickerstaff, V, et al. "Are multiple primary outcomes analysed appropriately in
randomised controlled trials? A review." Contemporary clinical trials 45 (2015):
8-12.
•
Phillips A., Haudiquet V. ICH E9 guideline ‘Statistical principles for clinical
trials’: a case study Stat. Med., 22 (1) (2003), pp. 1–11