Download Document

Critical appraisal of clinical
trials
Specific Types of Study
Randomised Controlled Trial (RCT)
• Population is randomly allocated to two groups
• One group is given a specific treatment or intervention
• On average the groups are identical because they are
randomised and therefore any difference in the measured
outcome is due to the intervention
• Specified follow up period and specified outcomes
• e.g. drug better than placebo; surgical procedure
compared with sham
Randomised Controlled Trial (RCT)
Advantages
• Allows rigorous evaluation of a single variable in a
previously defined population e.g. a new drug.
• Prospective i.e. collect the information after you decide
to do the study.
• Tries to disprove the null hypothesis
• Tries to eradicate bias because the two groups are
identical.
• Allows for meta-analysis later.
RCT’s are
• Unnecessary if:
• Clearly successful intervention
• Previous RCT’s or meta-analyses
• Impractical when:
• Unethical to randomise
• Large number needed
• Inappropriate when:
• Looking at prognosis
• Looking at validity of diagnostic tests
• Looking at quality of care
Background (1)
Practicing physicians must rely on the
literature to keep current on recent
developments on new therapies as well as
providing additional evidence on therapies
which have been long used in practice
Accurate reporting of a clinical trial is
important to aid the practicing physician in
deciding to adopt a new therapy or modify
therapies currently in use
Background (2)
• Proposals for requirements for reporting of
randomised trials
• JAMA 1994;272:1926-31
• Ann Intern Med 1994;121:894-5
• JAMA Editorial in 1995 suggests two groups
produce a unified statement
• Consolidated Standards of Reporting
(CONSORT)
• JAMA 1996; 276:637-9
Common Elements of
Reporting for All Trials
•
•
•
•
•
•
Population under study
Therapy details
Experimental design
Patient accounting
Quality control procedures
Statistical analysis
Special Reporting Requirements
• Non-randomized trials
• Randomized trials
The Term Prospective Trial Is a
“Pleonasm”
• “Trial” used increasingly in combination with
“prospective”
• “A PubMed search ‘prospective trial’ yielded
507 hits for the period 1999-2001. However,
‘prospective’ is a pleonasm (superfluous)”
• Cross-sectional trial!!
• “Because all trials are prospective by definition;
the only way to do a retrospective trial is for
the investigator to travel back in time with a
box of pills.”
• Letter in The Lancet, Sept 2002 by Martijn B Katan, Netherlands
Reporting in Clinical Trials
•
•
•
•
Describe the Plan
Report the Results
Confess to Problems
Interpret Objectively (no spin!)
Reporting in Clinical Trials
“The Published Paper”
1.
Identify clinical investigators and institutions
(experience, reputation)
2. Also, identify
• Sponsor (federal, industry)
• Data management team
• Statistical analysis team
Reporting in Clinical Trials
“The Published Paper”
3. Introduction & Background
a. Biochemical theory
b. Animal work
c. Phase I/II clinical studies
d. Previous large clinical studies
e. Other pharmaceutical analogues
Reporting in Clinical Trials
Definition of question
What was the primary question?
-
Clearly defined in advance
Reporting Clinical Trials
4. Methods Section
• Outcome variables
• Eligibility criteria
• Inclusion
• Exclusion
• Randomization Procedures
• Sample size justification
• Treatment & Control
Randomization
• Sequence generation
 Method used to generate the allocation
sequence, including any details about
restriction
• What does restriction mean?
• Allocation concealment
 Method used to implement the random
allocation sequence
• Implementation
 Who generated the allocation sequence, who
enrolled participants, and who assigned
participants to their groups
Allocation concealment
• A technique used to prevent selection bias by
concealing the allocation sequence from those
assigning participants to intervention groups,
until the moment of assignment.
Allocation concealment prevents
researchers from (unconsciously or
otherwise) influencing which participants
are assigned to a given intervention group.
Importance of allocation
concealment
• Unclearly concealed and inadequately
concealed trials, compared to
adequately concealed trials,
exaggerated the estimates of an
intervention’s effectiveness by 30% to
40%, on average.
Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence
of bias: dimensions of methodological quality associated with
estimates of treatment effects in controlled trials. JAMA
1995;273:408-412.
Reporting Clinical Trials
4. Methods Section (continued)
• Outcome assessment & blinding
• Measures of patient safety and adverse
events
• Predefined Subgroups
• Data monitoring plan
• Analysis Plan summary
Reporting in Clinical Trials
Treatment groups
a. Definitions
- Experimental
- Control
b. Dose escalation
c. Withdrawal for toxicity
Reporting in Clinical Trials
5.
Description of results
a.Full accounting of all patients entered on trials
- Completeness
- LTFU
- Withdrawal
b. Comparison of treatment groups as assigned
(ITT), baseline characteristics
c. Simple comparison of primary outcome
variables using means, proportions, graphs, with
measures of statistical precision (SE's, P-values)
d. Thorough analysis of side-effect data/adverse
effects
Reporting in Clinical Trials
5. Discussion
e.Adequate handling of the possible impact of
missing values, dropouts, non-adherence
f. Discussion, allowance for multiplicity in
number of interim analyses, number of endpoints
Reporting in Clinical Trials
5. Discussion
Consistency of results
a. among investigators and centers
b. Other independent studies of same drug or
analogues
c. Subgroups consistency
d. Primary and secondary outcomes
Reporting in Clinical Trials
6.
Conclusion
a.Brief summary
b.Strengths/weaknesses consistent with
data
c. Generalizability
d.Trade off in side effects - risk/benefit
THE LANCET
Double-blind trial of aspirin in primary prevention of
myocardial infarction in patients with stable chronic
angina pectoris
Clinical trials have demonstrated a prophylactic role for aspirin in myocardial
infarction and in unstable angina pectoris. The Swedish Angina Pectoris Aspirin
Trial (SAPAT) is the first prospective study of aspirin in stable angina.
“After showing good tolerance of sotalol for at
least three weeks the patients were randomised
double blind to aspirin 75 mg daily (n=1009) or
placebo (n=1026).”
Lancet 1992;340:1421-25
Obstetrics & Gynecology
Bacterial Vaginosis: Treatment with Topical
Intravaginal Clindamycin Phosphate
We tested topical intravaginal clindamycin phosphate at
ac
concentrations of 0.1, 1.0, and 2.0% in the treatment of
Alternative
62 women with symtomatic bacterial vaginosis in a propective, randomized, double-blind, placebo-controlled
efficacy
practitioners use systemic metronidazole and
common gastrointestinal side effects.
apy, particularly for pregnant women, is highly
able. This study examined the safety and
“. . . prospective, randomized,
double-blind, placebo-controlled
trial.”
Obstet Gynecol 1990;76:118-23.
British Journal of Obstetrics and Gynaecology
October 1991, Vol. 98, pp 980-987
(Patho)physiological Implications of Chronic Dietary
Sodium Restriction During Pregnancy; a longitudinal
prospective randomized study
Abstract: Objective--To study the possible pathophysiological implications of
long continued dietary sodium restriction in pregnancy
Design--Longitudinal prospective randomized study of the effects of a low
sodium diet compared with unrestricted sodium intake in pregnacy.
Setting--Academic Department of Obstetrics and Gynaecology at Sint Radboud
Hospital, Nijmegen, The Netherlands.
“Design - Longitudinal,
prospective randomized . . .”
Br J Obstet Gynaecol 1991;98:980-7
A Controlled Trial of Povidone-Iodine as Prophylaxis Against
Ophthalmia Neonatorum
Abstract Background. Neonatal conjunctivitis Ophthalmic neonatorum) continues to
cause blindness, because the agents used prophylactically to prevent this condition are
not completely effective and are not widely available in many parts of the world.
Povidone--iodine ophthalmic solution is an effective antibacterial agent with broad
antibacterial and antiviral activity to which no bacteria are known to be resistant, and it
is
“Randomization was achieved by
rotating the three medications after
each was used for a week.”
NEJM 1995;332:562-6
March 1991, Vol. 98, pp 260-264
Stopping Smoking in Pregnancy: Effect of a Selfhelp manual in Controlled Trial
Summary. For medical reasons, encouraging women to stop smoking
during pregnancy and post partum has high priority. Many smokers
want to stop smoking but decline clinical treatment when it is offered.
The aim of this study was to find a method which was accepted by a
large
number of smokers, had a high success rate and, at the same time,
“Women were randomized . . . born on days
1-10 of every month formed the control
group (n=231), and those born on days 1131 formed the treatment group (n=492). ”
Br J Obstet Gynaecol 1991; 98:260-4.
Obstetrics and Gynecology
Vol. 77, No. 3, March 1991
Nifedipine in the Treatment of Severe
Preeclampsia
We conducted a randomized clinical trial in which patients with severe
preeclampsia between 26-36 weeks of gestation receive either nifedipine (1030 mg sublingually, then 40-120 mg/day orally; N= 24) or hydralazine (6.2512.5 mg intravenously, then 80-120 mg/day orally; N= 25)
“We conducted a randomized controlled trial. .
.”
“Subjects were assigned to the nifedipine or
hydralazine group according to the week of the
month.”
Obstet Gynecol 1991; 77:331-7
Effectiveness of antibiotic prophylaxis
in preventing bacteriuria after
multichannel urodynamic
investigations: A blind, randomized
study in 124 female patients
Am J Obstet Gynecol;1991;165:679-81
On completion of the procedures, the
patients were randomly assigned to
prophylaxis or nonprophylaxis groups
according to hospital number. Both the
physician and the nurse technician were
blind as to which assignment the patient
received. Patients in group A received
nitrofurantoin 50 mg four times and
phenazopyridine hydrochloride 200 mg
three times for 1 day. Patients in group B
received phenazopyridine hydrochloride
only. The code was broken at the
completion of the study.
Randomization Process
Proper Approach
4 Major General
Medical Journals
Generation of Allocation
Sequence
49%
Allocation Concealment
26%
Both
15%
[Lancet 1990; 335: 149-153.]
CONSORT (1)
• “Intent is to make experimental process
more clear, flawed or not, so that users of
the data can more appropriately evaluate its
validity for their purposes”
• checklist
• figure
• available at www.consort-statement.org
CONSORT (2)
• Widely adopted by medical journals
• required by many from Jan 1, 1997
• Available in six languages
SPECIAL COMMUNICATION
The CONSORT Statement:
Revised Recommendations
for Improving the Quality of Reports
for Parallel-Group Randomized Trials
David Moher, MSc
Kenneth F. Schulz, PhD, MBA
Douglas Altman, DSc
for the CONSORT Group
JAMA, April 18, 2001—Vol 285, No. 15
1987
Dissemination
• Major general & internal medicine
journals endorsed CONSORT
 Required authors to submit RCT reports using
template
• Editorial groups that have endorsed
CONSORT
 World Association of Medical Editors (WAME)
 Council of Science Editors (CSE)
 International Committee of Medical Journal Editors
(ICMJE or Vancouver Group)
ACADEMIA AND CLINIC
The Revised CONSORT Statement for
Reporting Randomized Trials:
Explanation and Elaboration
Douglas G. Altman, DSc; Kenneth F. Schulz,
PhD; David Moher, MSc; Matthias Egger, MD;
Frank Davidoff, MD; Diana Elbourne, PhD; Peter
C. Gøtzsche, MD; and Thomas Lang, MA, for the
CONSORT Group
17 April 2001 Annals of Internal Medicine Volume 134 Number 8
663
Participant flow through a
randomized trial . . .
important?
Revamped web site
(www.consort-statement.org)
• Reprints of statement and E & E
document (copyright is in the public
domain)
• Data bank of “good” and “not so good”
examples of trial reporting
• Reference citations of new evidence
Does the CONSORT checklist
work? Methods
• 4 “general & internal medicine” journals
 3 endorsed CONSORT (BMJ, JAMA, Lancet)
 1 did not (NEJM)
• Hand searched all 4 journals for 1994 and
1998 (January to June), respectively
• Used number of CONSORT items, Jadad
score and adequacy of allocation
concealment
 Trained 2 assessors
Does CONSORT work?
More results
Total
Journal
BMJ
JAMA
Lancet
Total
Adopters
Total
NonAdopter
Unclear allocation
concealment
1994
N
1998
N
Pre
Mean (SD)
change
14
29
28
71
20
20
37
77
26
37
Change (CI)
(CI)
Pre
Mean
2.07 (0.92)
3.00 (1.04)
2.75 (0.89)
2.72 (1.00)
0.43 (-0.34, .20)
0.35 (-0.29, .99)
**
0.68 (0.14, .22)
***
0.45 (0.08, .82)
79
59
54
61
-29 (-62, 4)
-14 (-43, 16)
-24 (-48, 1)
****
-22 (-38, -6)
3.12 (1.03)
-0.01 (-0.55,0.54)
69
-8 (-33, 17)
Moher D, Jones A, Lepage L, for the CONSORT group. Use of the CONSORT
statement and quality of reports of randomized trials: a comparative before and
after evaluation? JAMA 2001;285:1992-1995.