Download Randomized Control Clinical Trial

Chapter 11
Issues in Analysis of
Randomized Clinical Trials
1
Issues in Analysis of
Randomized Clinical Trials
• Reference:
May, DeMets et al (1981)
Circulation 64:669-673
Peto et al (1976)
British Journal of Cancer
2
Sources of Bias
1. Patient selection
2. Treatment assignment
3. Patient Evaluation
4. Data Analysis
Methods to Minimize Bias
1. Randomized Controls
2. Double blind (masked)
3. Analyze what is randomized
3
What Data Should Be Analyzed?
• Basic Intention-to-Treat Principle
– Analyze what is randomized!
– All subjects randomized, all events during
follow-up
• Randomized control trial is the “gold”
standard”
• Definitions
Exclusions
– Screened but not randomized
– Affects generalizability but validity OK
Withdrawals from Analysis
– Randomized, but not included in data analysis
– Possible to introduce bias!
4
Patient Closeout
• ICH E9 Glossary
– “Intention-to-treat principle - …It has the
consequence that subjects allocated to a
treatment group should be followed up,
assessed, and analyzed as members of
that group irrespective of their compliance
with the planned course of treatment.”
5
Intention To Treat (ITT)
Principle
• Analyze all subjects randomized & all
events
• Beware of “look alikes”
– Modified ITT: Analyze subjects who get
some intervention
– Per Protocol: Analyze subjects who comply
according to the protocol
6
Patient Withdrawn in Analysis (1)
• Common Practice - 1980s
– Over 3 years, 37/109 trials in New England Journal of Medicine
published papers with some patient data not included
• Typical Reasons Given
a. Patient ineligible (in retrospect)
b. Noncompliance
c. Competing events
d. Missing data
7
Patient Withdrawn in Analysis (2)
A.
Patient INELIGIBLE
– After randomization, discover some patients did not in fact
meet entry criteria
– Concern ineligible patients may dilute treatment effect
– Temptation to withdraw ineligibles
– Withdrawl of ineligible patients, post hoc, may introduce
bias
8
Betablocker Heart Attack Trial
(JAMA, 1982)
• 3837 post MI patients randomized
• 341 patients found by Central Review to be ineligible
• Results
Eligible
Ineligible
Total
% Mortality
Propranolol
Placebo
7.3
9.6
6.7
11.3
7.2
9.8
Best
 In the ineligible patients, treatment works best
9
Acceptable Policies
For Ineligible Subjects
1. Delay randomization, confirm eligibility and allow
no withdrawals (e.g. AMIS) (Chronic Studies)
2. Accept ineligibles, allow no withdrawals
(e.g. BHAT, MILIS) (Acute Studies)
3. Allow withdrawals if:
a. Procedures defined in advance
b. Decision made early (before event)
c. Decision independent and blinded
d. Use baseline covariates only (two subgroups)
e. Analysis done with and without
10
B. WITHDRAWL FOR NON-COMPLIANCE
References: Sackett & Gent (1979) NEJM, p. 1410
Coronary Drug Project (1980) NEJM, p. 1038
•
Two Types of Trials
1. Management
- "Intent to Treat" Principle
- Compare all subjects, regardless of compliance
2. Explanatory
- Estimate optimum effect, understand mechanism
- Analyze subjects who fully comply
WITHDRAWALS FOR NON-COMPLIANCE
MAY LEAD TO BIAS!
11
Cancer Trial (5-FU & Radiation)
Gastric Carcinoma
• Reference: Moertel et al. (Journal of Clinical Oncology, 1984)
• 62 patients randomized
– No surgical adjuvant therapy
vs.
– 5-FU and radiation
• 5 year survival results
Randomized
Treatment
No Treatment
Percent (%)
23%
P < 0.05
4%
12
Cancer Trial (5-FU & Radiation)
Gastric Carcinoma
• According to treatment received 5 year
survival
Received
% Survival
Treatment
20%
Refused Treatment
30%
Control
4%
NS
13
Example: Coronary Drug Project
5-Year Mortality
Total (as reported)
By Compliance
< 80%
> 80%
Clofibrate
N
% Deaths
1103
20.0
1065
18.2
357
24.6
708
15.0
Placebo
N
% Deaths
2782
20.9
2695
19.4
882
28.2
1813
15.1
•
Adjusting for 40 covariates had little impact
•
Compliance is an outcome
Compliers do better, regardless of treatment
14
Example: Coronary Drug Project
2-Year Mortality
Compliance
Assessed
Total
< 80%
> 80%
Estrogen
N
% Deaths
903
6.2
488
6.1
415
6.3
Placebo
N
% Deaths
2361
5.7
436
9.9
1925
4.8
Comments
• Higher % of estrogens patients did not comply
• Beneficial to be randomized to estrogen & not take it
• (6.1% vs. 9.9%)
• Best to be randomized to placebo & comply (4.8%)
15
Example: Wilcox et al (1980) Trial, BMJ
6-Week Mortality
Propranolol
N
% Deaths
Total
132
7.6
Compliers
88
3.4
Non-compliers 44
15.9
Atenolol
N % Deaths
127
8.7
76
2.6
51
17.6
Placebo
N % Deaths
129
11.6
89
11.2
40
12.5
Comments
• Compliers did better than placebo
• Treatment non-compliers did worse than placebo
• Placebo non-compliers only slightly worse than compliers
• Analysis by compliers overestimates benefit
16
Aspirin Myocardial Infarction
Study (AMIS)
Compliance
Good
Poor
Total
% Mortality
Aspirin
Placebo
6.1
5.1
21.9
22.0
10.9
9.7
17
Summary of Compliance
• No consistent pattern
Example
Non-compliance Did Worse
AMIS
CDP Estrogen
Beta-blocker, Wilcox
Both Treatment & Control
Control Only
Two Treatments, Not Control
• Compliance an outcome, not always independent
of treatment
• Withdrawal of non-compliers can lead to bias
• Non-compliers dilute treatment
• Try hard not to randomize non-compliers
18
II. Competing Events
• Subject may be censored from primary event by some
other event (e.g. cancer vs. heart disease)
• Must assume independence
• If cause specific mortality used, should also look at total
death
• If non-fatal event is primary, should also look at total
death and non-fatal event
• Problem for some response measures
19
III. Problem of Definitions
•
Cause specific definitions hard to apply
•
Example: Anturane Reinfarction Trail (ART)
(NEJM, 1980)
Sudden Death
Classification
ART
Another Committee
Anturane
30/812
28/812
Placebo
48/817
39/817
P-value
0.03
0.17
20
Anturane Reinfarction Trial
Sudden Death
Category
Source Placebo
Anturane
P-value
All patients & all
NEJM 48/817
30/812
0.03
sudden deaths
AC
39/817
28/812
0.17
"Eligible" patients & NEJM 46/785
28/775
0.03
all sudden deaths
25/773
0.12
AC
37/782
• Problem of cause specific definitions
• AC = Another review committee
21
IV. "Wrong", Inconsistent,
Outlying Data
• "Wrong" or "outlying" data may in
fact be real
• Decisions must be made blind of
group assignment
• All modifications or withdrawals
must be documented
22
V. Missing Outcome Data
• Design with zero
– missingness may be associated with treatment
• for analysis, data are not missing at random
• even if same number missing, missing may be for
different reason in each treatment group
• Implement with minimum possible
• Analyze exploring different approaches
– if all, or most, agree, then more persuasive
23
“Best” and “Worst”
Case Analyses
Treatment Control
Total Events
170
220
Lost to Follow-up
30
10
"Best" Case
170
230
"Worst" Case
200
220
24
VI. Poor Quality Data
25
Poor Quality Data (1)
1. Lost to Follow-up
(enforced withdrawals)  NO DATA:
PROBLEMS:
– Not necessarily independent of
treatment
– Raises questions about study conduct
26
Poor Quality Data (2)
SOLUTIONS:
1. Keep to a minimum
• Easiest if vital status is the outcome
• Hardest if the response variables are
time-related measures requiring a
hospital or clinic visit
2. Censor at the time lost
– Can be done in survival analysis
– Assumes independence of treatment
27
Poor Quality Data (3)
SOLUTIONS:
3. Estimate missing data using previous
data or averages
4. “Best” case and “worst” case analyses
28
VII. Poor Clinic Performance in
a Multicenter Study
• If randomization was stratified by
clinic, then withdrawal of a clinic is
theoretically valid
• Withdrawal must be done independent
of the outcome at that clinic
29
Mortality in Aspirin Myocardial
Infarction Study (AMIS)
Aspirin
All 30 Centers
246/2267
7 “Selected” Centers
39
Placebo
P-value
219/2257
66
0.99
< 0.01
• In “selected” centers, aspirin showed superiority
30
Mortality in Beta-Blocker
Heart Attack Trial (BHAT)
Propranolol Placebo P-value
All 32 Centers
138/1916
188/1921 < 0.01
Cox adjusted Z = 3.05
6 “Selected” Centers 43
26
< 0.05
• In “selected” centers, propranolol worse
31
VIII. Special Counting Rules
• Events beyond a specified number of days after
treatment stopped not counted "non-analyzable"
• Examples
1.
2.
"7 Day Rule"
"28 Day Rule"
Anturane (1978) NEJM
Timolol (1981) NEJM
• If used, must
– Specify in advance
– Be a long period to insure termination not related to
outcome
– Analyze results both ways
32
IX. Fishing or
Dichotomizing Outcomes
• Common practice to define a response
(S,F) from a non-dichotomous variable
• By changing our definition, we can alter
results
• Thus, definitions stated in advance
• Definitions should be based on external
data
33
Dichotomizing Outcomes
Example
Subject
1
2
...
25
Mean
Pre
72
74
Trt A
Post
72
73

0
1
73
73
0
74.0
73.2 0.8
Heart Rate
Trt B
Pre
Post
72
70
71
68
79
79
74.4
74.0

2
3
0
0.4
34
Three Possible Analyses (1)
Change 
1. F = < 7
S=>7
Treatment A
23
2
Treatment B P-Value
25
0.49
0
35
Three Possible Analyses (2)
Change 
1. F = < 7
S=>7
2. F = < 5
S=>5
Treatment A
23
2
19
6
Treatment B P-Value
25
0.49
0
25
0
0.02
36
Three Possible Analyses (3)
Change 
1. F = < 7
S=>7
Treatment A
23
2
Treatment B P-Value
25
0.49
0
2. F = < 5
S=>5
19
6
25
0
0.02
3. F = < 3
S=>3
17
8
18
7
0.99
37
X. Time Dependent Covariate
Adjustment
• Classic covariate adjustment uses baseline
prognostic factors only
– Adjust for Imbalance
– Gain Efficiency
• Adjustment by time dependent variates not
recommended in clinical trials (despite Cox
time dependent regression model)
• Habit from epidemiology studies
38
Coronary Drug Project
5-Year Mortality
Example
Baseline
Cholesterol
< 250mg%*
< 250
> 250 mg%
> 250 **
•
•
Cholesterol
Change
Fall
Rise
Fall
Rise
% Deaths
Clofibrate Placebo
16.0
21.2
25.5
18.7
18.1
20.2
15.5
21.3
Little change in placebo group
Best to have
a.
Low cholesterol getting lower *
b.
High cholesterol getting higher **
39
Example: Cancer Trials
• A common practice to compare survival
on patients with a tumor response
• Problem is that patient must first
survive to be a responder
length - bias sampling
40
Cancer Trials (1)
Advanced Breast Cancer: Surgery vs. Medicine
Santen et al. (1981) NEJM
(Letter to editor, Paul Meier, U of Chicago)
• A randomized clinical trial comparing
surgical adrenalectomy vs. drug therapy
in women with advanced breast cancer
• 17 pts withdrawn from surgery group
10 pts withdrawn from medical group
41
Cancer Trials (2)
• Reasons
– Medical group (10 pts)
2 stopped taking their drugs
5 drug toxicity
– Surgical group (17 pts)
7 later refused surgery
8 rapid progression precluding surgery
• No follow-up data on these 27 pts
presented
42
XI. Subgroup Analyses
43
False Positive Rates
The greater the number of subgroups analyzed
separately, the larger the probability of making
false positive conclusions.
No. of Subgroups
1
2
3
4
5
10
False Positive Rate
.05
.08
.11
.13
.14
.19
44
Subgroup Analyses
• Focusing on a particular
“significant” subgroup can be risky
– Due to chance
– Results not consistent
• Estimates not precise due to small
sample size
45
MERIT Total Mortality
46
MERIT
47
MERIT
(AHJ, 2001)
48
Praise I
Ref: NEJM, 1996
•
•
•
•
•
•
Amlodipine vs. placebo
NYHA class II-III
Randomized double-blind
Mortality/hospitalization outcomes
Stratified by etiology (ischemic/non-ischemic)
1153 patients
49
PRAISE I
50
PRAISE I - Interaction
• Overall P = 0.07
• Etiology by Trt Interaction
P = 0.004
• Ischemic P = NS
• Non-Ischemic P < 0.001
51
PRAISE I - Ischemic
52
PRAISE I – Non- Ischemic
53
PRAISE II
•
•
•
•
•
•
Repeated non-ischemic strata
Amlodipine vs. placebo
Randomized double-blind
1653 patients
Mortality outcome
RR  1.0
54
Three Views
• Ignore subgroups and analyze only by
treatment groups.
• Plan for subgroup analyses in
advance. Do not “mine” data.
• Do subgroup analyses
However view all results with caution.
55
Analysis Issues Summary
• Important not to introduce bias into
the analysis
• ITT principle is critical
• Important to have “complete” followup
• Off treatment is not off study
56