Download Conventional Statistical Rules

CTOS 2012 - Prague
BAYESAN STATISTICS
FOR CLINICAL
INVESTIGATORS
Paolo Bruzzi
National Institute for Cancer Research
Genoa - Italy
“Bayesian”?
• Predictive value of diagnostic tests
• Studies of expression profiles (micorarrays)
• Trials with a Bayesian component
– Interim analyses
– Bayesian design
Differences between
Conventional and Bayesian
Statistics
• Meaning of probability
• Use of prior evidence
Conventional P
Probability of an observation
Bayesian Probability
Probability of a hypothesis
Conventional P
Probability of observing what was actually
observed if H0 true
Bayesian Probability
Probability of H0/H1/H2/H3… given
observed data (and prior distribution)
Conventional P
Probability of the observed difference (if the
experimental therapy does not work)
Bayesian Probability
Probability that the experimental therapy
works/doesn’t work (given observed
difference and prior knowledge)
Examples - Diagnosis
Mr. XY shows a lung nodule at TC
• Frequentist
Probability that Mr. XY shows a lung nodule
(if he doesn’t have lung cancer)
• Bayesian
Probability that Mr. XY has lung cancer
(given his nodule and his prior risk)
Examples - Diagnosis
Mr. XY shows a lung nodule at TC
• Frequentist
Probability that Mr. XY shows a lung nodule
(if he doesn’t have lung cancer)
• Bayesian
Probability that Mr. XY has lung cancer
(given his nodule and his prior risk)
Examples – Clinical trial
Control therapy (CT):
5/40 responses
Experimental therapy (ET): 10/40 responses
• Frequentist
Probability to observe 10/40 vs 5/40 if CT
and ET are identical
• Bayesian
Probability that CT and ET are identical
(given 5/40, 10/40 and prior knowledge)
Examples – Clinical trial
Control therapy (CT):
5/40 responses
Experimental therapy (ET): 10/40 responses
• Frequentist
Probability to observe 10/40 vs 5/40 if CT and
ET are identical P= 0.15 Not significant
Ho (CT & ET identical) NOT REJECTED
Examples – Clinical trial
Control therapy (CT):
5/40 responses
Old drugs, new schedule 10/40 responses
• Frequentist
Probability to observe 10/40 vs 5/40 if CT and
ET are identical P= 0.15 Not significant
Ho (CT & ET identical) NOT REJECTED
Examples – Clinical trial
Control therapy (CT):
5/40 responses
Very Promising ther. (ET): 10/40 responses
• Frequentist
Probability to observe 10/40 vs 5/40 if CT and
ET are identical P= 0.15 Not significant
Ho (CT & ET identical) NOT REJECTED
Examples – Clinical trial
Control therapy (CT):
5/40 responses
Experimental therapy (ET): 10/40 responses
• Bayesian
Probability that CT and ET are identical ?
(given 5/40, 10/40 and prior knowledge)
It depends on prior knowledge!
Examples – Clinical trial
Control therapy (CT):
5/40 responses
Herbal + standard (ET): 10/40 responses
• Bayesian
Probability that CT and ET are identical
(=herbal therapy not effective)?
Still high!
Examples – Clinical trial
Control therapy (CT):
5/40 responses
Very Promising ther. (ET): 10/40 responses
• Bayesian
Probability that CT and ET are identical
(that is, new therapy not effective)?
Quite low!
NOTE
The different meaning of Bayesian probability
in theory, has
• Implications for (clinical) decision analysis
• No peculiar implications for rare tumors
Differences between
Conventional and Bayesian
Approaches
• Meaning of probability
• Use of prior evidence
Conventional P
Probability of the observed difference (if the
experimental therapy does not work)
Bayesian Probability
Probability that the experimental therapy
works/doesn’t work (given observed
difference and prior knowledge)
Conventional Statistical
Reasoning
1. Starting hypothesis (H0):
new treatment = standard one
Conventional Statistical
Reasoning
1. Starting hypothesis (H0):
new treatment = standard one
2. To demonstrate: new treatment >> standard,
reject null hypothesis
Conventional Statistical
Reasoning
1. Starting hypothesis (H0):
new treatment = standard one
2. To demonstrate: new treatment >> standard,
reject null hypothesis
3. To this purpose, only evidence collected
within one or more trials aimed at falsifying it
can be used
Conventional Statistical
Reasoning
1. Starting hypothesis (H0):
new treatment = standard one
2. To demonstrate: new treatment >> standard,
reject null hypothesis
3. To this purpose, only evidence collected
within one or more trials aimed at falsifying it
can be used -> LARGE SAMPLE SIZE
Conventional Statistical
Reasoning
To this purpose, only evidence collected within
one or more trials aimed at falsifying it can
be used -> LARGE SAMPLE SIZE
No use of
– External evidence
– Evidence in favor of…
Example
Question: Efficacy of radiochemotherapy in a
tumor type very rare in a site (e.g.
squamous histology in stomach c.)
External evidence: RX+CTX is effective in
squamous cancers in more common sites
Evidence in favor of..: The observed
response rate is very high (e.g. 6/10)
Does this information affect ….
- the sample size of the phase III
trial aimed to assess RT+CTX in
squamous gastric c. ?
- the analysis of its results (p value)?
Squamous gastric cancer
Planning a trial of
RT+CTX
Analysing its results
(p value)
Squamous gastric cancer
Planning a trial of
RT+CTX
Herbal therapy
Analysing its results
(p value)
Squamous gastric cancer
Planning a trial of
RT+CTX
Herbal therapy
Analysing its results
(p value)
Conventional (frequentist)
statistical reasoning
Exclusive reliance on experimental evidence
Large Trials
Large Trials
Implication in rare tumors:
Generic Selection criteria
(All STS’s + Stage + treatment line)
- Appropriate for chemotherapy trials
- Possibly inappropriate for trials of Targeted
Therapies
Conventional (frequentist)
statistical reasoning
Experimental evidence
Bayesian statistical reasoning
Experimental evidence
+ Previous Knowledge
Bayesian Approach
Prior
Evidence
+
Experimental
Evidence
Posterior Probability Distribution
Conventional Probability
Probability of a positive test given disease/no
disease (Sensitivity, specificity)
Bayesian Probability
Probability of disease given test result and
disease prevalence (Predictive value)
Previous Knowledge?
• Biological rationale
• Evidence of activity
• Efficacy in other diseases with
similarities
• Efficacy in other stages of the same
disease
Prior evidence in Bayesian statistics
• Needed in order to compute posterior
probability
Prior evidence in Bayesian statistics
• Needed in order to compute posterior
probability
• It must be transformed into a probability
distribution (shape, mean, median,
standard deviation, percentiles, etc)
Prior evidence in Bayesian statistics
• Needed in order to compute posterior
probability
• It must be transformed into a probability
distribution
• Based on
– Objective information
– Subjective explicit beliefs
– Both
Prior evidence in Bayesian statistics
• Note: The difference between Bayesian and
conventional statistics decreases with
increasing strength of the empirical
evidence
Prior evidence in Bayesian statistics
Difference between Bayesian and
conventional statistics
Young man, never smoked, no family history
Prior probability of lung c.=1/100.000
- Nodule at routine x-ray
- Suspicious lesion at TC
- Histological confirmation after biopsy
Posterior Probability?
Prior evidence in Bayesian statistics
Difference between Bayesian and
conventional statistics
Epidermoid lung cancer: Palliative care
prolongs survival: A priori: 1-30%
Small monocentric trial: HR= 0.75 (0.55-0.95)
Large multicentric trial: HR= 0.8 (0.7 -0.9)
Meta-analysis of 8 trials:HR= 0.82 (0.76-0.88)
Prior evidence in Bayesian statistics
Frequent tumors
Prior direct evidence Estimates
Indirect evidence?
Empirical evid.
Prior evidence in Bayesian statistics
Rare tumors
Prior direct evidence Estimates
?
Indirect evidence!
Empirical evid.
Need to use all the available
information
-Direct evidence
-Experimental
-Prior
-Indirect evidence
Example
Mortality
Tumor X
Nil vs A 15% vs 12.5%
N=12000
P = 0.0007
H0 Rejected: A is effective in X
Example
Mortality
Tumor X
Nil vs A 15% vs 12.5%
N=12000
P = 0.0007
Tumor Y
N= 240
Nil vs A
15% vs 7.5%
P=0.066
H0 not rejected: A not shown effective in y
Prior Information:
Mortality
Tumor X
Nil vs A 15% vs 12.5%
N=12000
P = 0.0007
Tumor Y
N= 240
Nil vs A
15% vs 7.5%
P=0.066
Prior Information:
X and Y are BRAF+
Mortality
Tumor X
Nil vs A 15% vs 12.5%
N=12000
P = 0.0007
Tumor Y
N= 240
Nil vs A
15% vs 7.5%
P=0.066
Prior Information:
X and Y are BRAF+
A = Anti BRAF
Mortality
Tumor X
Nil vs A 15% vs 12.5%
N=12000
P = 0.0007
Tumor Y
N= 240
Nil vs A
15% vs 7.5%
P=0.066
Prior Information:
X and Y are BRAF+
A = Anti BRAF
Mortality
Tumor X
Nil vs A 15% vs 12.5%
N=12000
P = 0.0007
Tumor Y
N= 240
Nil vs A
15% vs 7.5%
P=0.066
INTERPRETATION?
A different example
1. Study Backgound
Uterine Sarcomas Stage I-II
- High rate of distant relapses
- Adjuvant pelvic RT in US: No impact on OS
- Adjuvant CTX: Scanty and inconsistent data:
3 cycles of + RT compared with an historical
control group of RT alone:
3-year DFS: RT alone= 43% CT+RT = 76%
2. Study Aims
…a randomized trial to confirm or not a
benefit in terms of PFS and OS.
…a multicentric phase III study, comparing API
chemotherapy regimen followed by RT versus
RT alone for patients with localized US after
complete surgery.
3. Statistical considerations
• Primary Endpoint: 3-yrs EFS (OS
secondary)
• Sample Size: 256 patients (128 in each
arm)
- 80% power
- Delta = 20% diff. in EFS (from 35% to
55%)
- 2-sided 5% significance level
4. Results
• In 8 years (October 2001- July 2009)
81 patients randomized in 19 institutions
“Study was stopped because of lack of
recruitment”
• 3-year DFS = 55% vs 41% (P = 0.048)
• 3 years OS = 81% in arm A vs 69% (P= 0.41)
Conclusions
“We have shown for the first time a statistical
impact of adjuvant chemotherapy on DFS in
this population of 81 patients without
impact on OS yet.”
Conventional analysis
• 3-year DFS = 55% vs 41% (P = 0.048)
Probability of observing this difference if CT
not effective (H0) on DFS = 4.8% -> Reject
H0
• 3 years OS = 81% vs 69% (p=0.41)
Probability of observing this difference if CT
not effective (H0) = 41% -> “No
difference” in OS
Bayesian Analysis of OS
Background:
a) Expected effect on DFS: 20%
improvement
b) Any effect on OS mediated by the effect
on DFS
c) Observed effect on DFS = 14%
improvement in 3-year DFS (55% vs
41%)
LIKELY EFFECT ON OS?
Predicted and Observed Effect of CT on
OS
Endpoint
Delta (95% CI)
EFS
14% (1% to 27%)
Predicted Effect on OS?
-20% -10% 0
RT better
+10% +20% +30% +40% +50%
CT better
Predicted and Observed Effect of CT on
OS
Endpoint
Delta (95% CI)
EFS
14% (1% to 27%)
Observed Effect on OS
OS
12%
-20% -10% 0
RT better
+10% +20% +30% +40% +50%
CT better
Predicted and Observed Effect of CT on
OS
Endpoint
Delta (95% CI)
EFS
14% (1% to 27%)
However….
OS
12% (-18% to 42%)
-20% -10% 0
RT better
+10% +20% +30% +40% +50%
CT better
Bayesian Analysis of OS
a) Expected effect on DFS: 20% improvement
b) Any effect on OS mediated by the effect on
DFS
c) Observed effect on DFS = 14%
improvement in 3-year DFS (55% vs 41%)
LIKELY EFFECT ON OS?
Likely effect on OS?
Needed step: Make assumptions on the type and
strength of the association between DFS and
OS
Assumptions used (based on Adjuvant studies in
Breast c. and Colon c.):
1. Effect on OS 20%weaker than effect on DFS
2. Weak correlation (R2 = 0.50)
3. (Sensitivity analyses)
Bayesian Analysis of OS
a) Expected effect on DFS: 20% improvement
b) Any effect on OS mediated by the effect on
DFS
c) Observed effect on DFS = 14%
improvement in 3-year DFS (55% vs 41%)
LIKELY EFFECT ON OS =
Prior Probability distribution: 5-14%
improvement
Bayesian Analysis of OS
Prior Probability distribution:
5-14% improvement
+
Observed Effect on OS: 12% improvement
=
Posterior Probability of effect on OS
Predicted and Observed Effect of CT on
OS
Endpoint
Delta
EFS
14% (1% to 27%)
OS
(95% CI)
12% (-18% to 42%)
-20% -10% 0
RT better
+10% +20% +30% +40% +50%
CT better
Predicted and Observed Effect of CT on
OS
Endpoint
Delta (95% CI)
EFS
14% (1% to 27%)
OS
Credible Effect on
OS
10% (1% to 20%)
-20% -10% 0
RT better
+10% +20% +30% +40% +50%
CT better
Predicted and Observed Effect of CT on
OS
Endpoint
Delta (95% CI)
EFS
14% (1% to 27%)
OS
NOTE
10% (1% to 20%)
-20% -10% 0
RT better
+10% +20% +30% +40% +50%
CT better
Hazard Ratios for adjuvant
therapies
Endpoint
CT +RT
RT
HR
0.67
0.56
0.54
French Study, Sarcomas
3-yrs DFS
55%
41%
3-yrs OS
81%
69%
5-yrs OS
72%
55%
Breast c
OS
Colon c.
OS
0.6-0.8
0.7-0.9
Conclusions of the study
• Conventional analysis: Significant effect on
EFS, no effect on OS
• Bayesian analysis: a remarkable effect on
EFS (10-20% absolute improvement) and
on Overall Survival (5-15% absolute
improvement)
Conclusion of the presentation
In Rare tumors
- Limited prior direct evidence
- Limited experimental evidence
IT IS NECESSARY TO USE BAYESIAN
STATISTICS
BAYESIAN STATISTICS
- Not a mysterious statistical religion for
adepts who trust in it
- Not a confortable container where
everyone can accomodate his crazy beliefs
-Not a sneaky statistical machinery to fool
referees and regulatory agencies
BAYESIAN STATISTICS
It is simply
- a tool to reproduce human reasoning,
- by connecting decisions to knowledge,
- in a transparent fashion