Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Stan Letovsky Senior Director, Computational Sciences Costs and Benefits of Biomarkers in Clinical Trials Washington D.C. September 29, 2006 © 2006 Millennium Pharmaceuticals Inc. Drug Response/Toxicity Biomarkers • Biomarker is a measurement or test on a patient that can predict (with some probability) – Efficacy of a treatment – Toxicity of a treatment – Disease severity (independent of drug) • E.g. Gleevec/BCR-ABL, Iressa/EGFRmut • Drug-specific biomarkers need to be validated in clinical trials to affect approvals. ©2006 Millennium Pharmaceuticals, Inc. 2 ©2006 Millennium Pharmaceuticals, Inc. 3 Question Under what circumstances does it make sense to include a biomarker efficacy hypothesis as part of the main study objectives of a clinical trial? • What are the costs? – Assays, logistics – P-value / sample-size adjustments • What are the benefits? – Increased probability of drug approval ©2006 Millennium Pharmaceuticals, Inc. 4 Possible Trial Designs • Traditional – efficacy only, no biomarker component • Biomarker Discovery – hitchhike on phase 2-3 trial, resulting biomarkers not validated. • Static Biomarker trial – specific biomarker hypotheses tested as part of trial design, could yield validated biomarkers and stratified market. Patient population not biased by biomarker. • Adaptive Validation – a form of adaptive trial in which a biomarker hypothesis is formulated at an interim point. May yield a validated biomarker. No biased sampling. • Adaptive Sampling – a form of adaptive trial in which a biomarker hypothesis is evaluated at an interim point, and subsequent patient selection may be biased by the biomarker. – for Response: Sampling biased towards responding subset / away from adversely-responding subset – for Speed: Sampling biased towards severest disease for faster trial. – for Power: Sampling is biased to allocate more sample to the hypothesis that is most likely to benefit. ©2006 Millennium Pharmaceuticals, Inc. 5 Multiple Comparison Corrections • Study Design#1: – Hypothesis H: “drug not efficacious” • Significance threshold a=.05 • Study Design#2: – Hypothesis H0: “drug not efficacious” • Significance threshold a=.04 – Hypothesis H1: “drug not efficacious in biomarker positive population” • Significance threshold a=.05-.04=.01 ©2006 Millennium Pharmaceuticals, Inc. 6 Power Curves for Static Design (schematic) $$ For a given choice of a (significance) and b (power) get curve of N vs F. $ 6.8% for a1=.01!! n=Max affordable study size or duration N= Sample Size H0 powered at a0 <a : (f0>f , N0) H powered at a: (f,,n) Hi powered at ai : (fi>f0 , Ni=N0*pi) Adding biomarker hypothesis imposes a multiple comparisons “tax” that must be paid in dollars (by increasing sample size), sensitivity (increasing F) or risk (decreasing power). or Min clinically acceptable effect f f0 f1 F = Effect size (e.g. TTP for new drug + SOC / SOC alone) ©2006 Millennium Pharmaceuticals, Inc. 7 F0>=F1*p1 Parameter Space for Static Design Biomarker Win: Reject H1 only: biomarker pays off; stratified market better than none. Payoff=p1 vs. 0 F1 = mean effect size f in biomarker 1 population Must have f1*p1 < f for biomarker strategy to be viable. The steeper the line, the smaller market. F0<F1 Impossible to be left of blue line Drug Failure: Possible Partial Backfire: Possible Partial Backfire#2: Apparent success of H0 explained by H1 Reject H1 only, would have rejected H0 w/o biomarker. Market may be Trial outcome is stratified; a point1 in payoff=p or the 1 vs. 1. 1 plane F0,F Biomarker Backfire: Reject none, drug is no good, biomarker didn’t help. Fail to reject H0 Slope line have is if butof would biomarker you hadn’t used enrichment B1 the biomarker. Total loss of market. Payoff = 0 vs. 0 Redundant: Reject both: didn’t need biomarker. Biomarker not predictive on green line, antipredictive below Payoff = 1 vs. 1 Biomarker Failure: Reject H0 only; biomarker useless, no harm done. Payoff=1 vs. 1 Payoff = 0 vs. 1 f1*p1 f f0 F0 = mean effect size in entire study population ©2006 Millennium Pharmaceuticals, Inc. 8 Likelihood: Outcomes Are Not Equally Probable Joint Distribution of F0 X F1 Implied Distribution of F 1 Given prior pdf for F0 (e.g. from phase II results, literature) and B1, (made up), can infer (assuming independence) joint distribution of F0 X F1 and pdf of F1. 1.5 1.50 1 1.00 0.5 0.50 NB: F=T/C. 0 Biomarker Enrichment 1 Pr(F1=x) 0 0.00 0 Pr(F0=x) Pr(B i=x) 4 2 0 0.5 1 1.5 Prior on B 1 2 2.5 ©2006 Millennium Pharmaceuticals, Inc. 0.5 1 1.5 4 2 0 0 0.5 1 1.5 Prior on F0 from phase II 9 Utility: Outcomes Are Not Equally Valuable Joint Distribution of F 0 X F1 NPV1 50 1.50 e2 1.00 e1 0.50 0.00 0 e0 0.5 1 0.8 30 0.6 20 0.4 10 0.2 1.5 10 EPV1 10 40 8 30 20 4 20 10 2 10 30 40 50 ©2006 Millennium Pharmaceuticals, Inc. 40 50 -3 x 10 5 40 6 20 30 50 30 10 20 EPV1 - EPV 0 -3 x 10 50 = X 40 0 -5 10 20 30 40 50 10 A Biomarker-Favorable Scenario Joint Distribution of F 0 X F1 NPV1 50 2.00 40 0.8 30 0.6 20 0.4 10 0.2 1.50 1.00 0.50 0.00 0 0.5 1 1.5 EPV1 2 10 30 40 50 EPV1 - EPV 0: =28% -3 x 10 50 20 -3 x 10 50 40 4 30 3 30 20 2 20 10 1 10 10 20 30 40 50 5 40 0 -5 10 20 30 40 50 If unlikely to succeed in main trial, but likely in biomarker subpopulation. Better redesign trial? ©2006 Millennium Pharmaceuticals, Inc. 11 Parsing the Parameter Space • Simply by assuming reasonable values of f,f0,f1 and looking at different plausible priors one can learn a lot: – If the F0 prior makes it likely that F0 > f1, there is no need to bother with a biomarker. – If it is likely that F0 > f but it may not be > f1, you may be better off not risking the multiple comparison “tax”. – If there is substantial risk that F0 < f and you have a biomarker with substantial likelihood of significant enrichment, the biomarker strategy may have higher EPV. ©2006 Millennium Pharmaceuticals, Inc. 12 Multiple Comparison Tax Relief • Suppose regulator wants to encourage biomarker validation… • What is consequence of ignoring a=.01 worth of multiple comparison correction to main efficacy hypothesis? – No change to drug approvals in main study population – false positive rate of 5% already deemed societally acceptable. – 1% Probability of false positive “biomarker wins” already deemed acceptable in 4%/1% split. – Assuming something like 10% of biomarkers tested really are predictive, precision=91%, FDR=9%. – Social cost of biomarker backfire avoided ©2006 Millennium Pharmaceuticals, Inc. 13 Adaptive Biomarker Validation good Initial Unbiased Recruiting Interim Evaluation Of Biomarker Add Biomarker Hypothesis To Trial Design No good Continue As Before Advantages: •Can validate biomarker during phase III Disadvantages: •Never been done, breaking new regulatory ground •Some complex statistical issues – bias, multiple comparisons… ©2006 Millennium Pharmaceuticals, Inc. 14 E.g. Freidlin and Simon Adaptive Signature Design, Clinical Cancer Research Vol. 11, 7872-7878, Nov 2005 Biomarker-driven Adaptive Sampling good Initial Unbiased Recruiting Interim Evaluation Of Biomarker Recruit Biomarker Positive Population No good Continue Normal Recruiting Advantages: •Can validate biomarker during phase III •If biomarker works, save money and/or improve chances of approval Disadvantages: •Never been done, breaking new regulatory ground •Some complex statistical issues – bias, multiple comparisons… ©2006 Millennium Pharmaceuticals, Inc. 15 Parameter Space View of Adaptive Validation Interim outcome gives estimate of final outcome Uncertainty radius varies inversely with interim sample size Interim outcome is a point in the F0,F1 plane f1 Adaptive strategy is triggered if interim point falls in a predefined region. Decision analysis optimizes shape of region. Want final point in same (or better) region as interim point. f ©2006 Millennium Pharmaceuticals, Inc. f0 16 Conclusions • The requirement of correcting for multiple comparisons has a significant impact on the incentives for including biomarkers in clinical trial designs. • The circumstances under which a cost/benefit analysis favors inclusion of a biomarker hypothesis in the main study objectives may be surprisingly rare. • Adaptive designs combining biomarker discovery, validation and use warrant further investigation. ©2006 Millennium Pharmaceuticals, Inc. 17 Acknowledgements Millennium • Mark Chang • Barb Bryant • Chris Hurff • Bill Trepicchio • Andy Boral FDA (CDER) • Gene Pennello ©2006 Millennium Pharmaceuticals, Inc. 18 SM Breakthrough science. Breakthrough medicine. ©2006 Millennium Pharmaceuticals, Inc.