Download Joan-Ramon Laporte

The golden standard or fool’s gold
Selective data, impact on safety
Clinical trials on trial
HAI Europe Open Seminar 2008
Neues Stadthaus, Berlin, 21 November 2008
Joan-Ramon Laporte
RCTs – The golden standard or fool’s gold?
Internal validity
External validity - Efficacy vs Effectiveness
Publication bias
Fraud
Internal validity
• Did the control group receive optimal treatment?
Internal validity
• Did the control group receive optimal treatment?
• Was the dose of the control group adequate?
≤ 12 mg haloperidol
> 12 mg haloperidol
-0.5
-0.4
-0.3
-0.2
-0.1
Favours
atypical
0
0.1
Favours
haloperidol
Drop out rates by dose of comparator drug in trials of patients
with schizophrenia or related disorders (risk difference and
95 % confidence intervals)
Geddes et al., 2000
Internal validity
• Did the control group receive optimal treatment?
• Was the dose of the control group adequate?
• Was the sample size adequate to identify any
relevant difference?
Internal validity
• Did the control group receive optimal treatment?
• Was the dose of the control group adequate?
• Was the sample size adequate to identify any
relevant difference?
• Did the published results refer to the primary variable?
Internal validity
• Did the control group receive optimal treatment?
• Was the dose of the control group adequate?
• Was the sample size adequate to identify any
relevant difference?
• Did the published results refer to the primary variable?
• Have all the trial results been published?
Internal validity
• Did the control group receive optimal treatment?
• Was the dose of the control group adequate?
• Was the sample size adequate to identify any
relevant difference?
• Did the published results refer to the primary variable?
• Have all the trial results been published?
• Were the results presented as a relative risk reduction,
or as an absolute risk reduction?
RCTs – The golden standard or fool’s gold?
Internal validity
External validity - efficacy vs effectiveness
Publication bias
Fraud
External validity of clinical trials
•
•
•
•
•
•
•
•
•
Context
Reference population
Selection criteria
Diagnostic criteria
Follow up
Treatments (doses, compliance)
Duration
Primary and other variables
Adverse effects
Lancet 2005; 365: 82-93
(Un)transferability to clinical practice
Patients in RCTs differ from those in real practice:
– Age
– Comorbidity
– Other treatments
– Doses taken, compliance
– Diagnostic criteria
Efficacy vs effectiveness
RCT
Nº of patients
Duration
Populations
Comorbidity
Conditions
Nº of drugs
Dose/dosage
Pattern of use
Follow up
102-103
Short
High risk groups
excluded
Generally excluded
Well defined
One or limited
Generally constant
Continuous
Careful
UCP
104-107
Longer
Potentially the
whole population
Often present
Ill-defined
Undetermined
Often variable
Intermittent
Less careful
RCTs – The golden standard or fool’s gold?
Internal validity
External validity - efficacy vs effectiveness
Publication bias
Fraud
RCTs – The golden standard or fool’s gold?
Internal validity
External validity - efficacy vs effectiveness
Publication bias
Fraud
Third Report of the Expert Panel on Detection,
Evaluation, and Treatment of High Blood Cholesterol
in Adults (Adult Treatment Panel III) (ATP III)
Financial Disclosure: Dr Grundy has received honoraria from Merck, Pfizer, Sankyo,
Bayer, and Bristol-Myers Squibb. Dr Hunninghake has current grants from Merck,
Pfizer, Kos Pharmaceuticals, Schering Plough, Wyeth Ayerst, Sankyo, Bayer,
AstraZeneca, Bristol-Myers Squibb, and G. D. Searle; he has also received consulting
honoraria from Merck, Pfizer, Kos Pharmaceuticals, Sankyo, AstraZeneca, and Bayer. Dr
McBride has received grants and/or research support from Pfizer, Merck, Parke-Davis,
and AstraZeneca; has served as a consultant for Kos Pharmaceuticals, Abbott, and
Merck; and has received honoraria from Abbott, Bristol-Myers Squibb, Novartis, Merck,
Kos Pharmaceuticals, Parke-Davis, Pfizer, and DuPont. Dr Pasternak has served as a
consultant for and received honoraria from Merck, Pfizer, and Kos Pharmaceuticals, and
has received grants from Merck and Pfizer. Dr Stone has served as a consultant and/or
received honoraria for lectures from Abbott, Bayer, Bristol-Myers Squibb, Kos
Pharmaceuticals, Merck, Novartis, Parke-Davis/Pfizer, and Sankyo. Dr Schwartz has
served as a consultant for and/or conducted research funded by Bristol-Myers Squibb,
AstraZeneca, Merck, Johnson & Johnson-Merck, and Pfizer.
Conclusions
• The RCT is the best epidemiological method
for causal inference
• However, it is often performed in a way
which favours the sponsor’s treatment:
– In its design
– In data analysis and interpretation
– In the publication of results
• At best, RCTs are one of many pieces of
evidence about therapeutic interventions
Conclusions
• The appraisal of innovation should not only take
into account the so-called EBM, but also other
evidence:
– Pharmacodynamics
– Pharmacokinetics
– Availability of therapeutic alternatives
• The medical literature is no longer reliable for valid
information
• Research should be performed, analyzed and
published in a way which should be independent
from commercially interested parties