Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Demonstrating that an HIV Vaccine Lowers the Risk and/or Severity of HIV infection D.Mehrotra1*, X.Li1, P.Gilbert2 1Merck 2Fred Research Laboratories, Blue Bell, PA Hutchinson Cancer Research Center & Univ. of Washington, Seattle, WA Adapted from Mehrotra’s talk at ENAR, Austin, TX March 21, 2005 Biostat 578A Lecture 6 Outline • • • • • HIV vaccine POC efficacy trial BOI vs. Simes’ method Adjusting for selection bias BOI vs. adjusted Simes’ method Concluding remarks 2 HIV Vaccine POC Efficacy Trial • Design - Randomized, double-blind, multinational trial - MRKAd5 gag/pol/nef versus placebo (1:1) - 1500 subjects at high risk of becoming HIV+ - Continue until 50 events (HIV infections) accrue • Co-Primary Endpoints - HIV infection status (infected/uninfected) - Viral load set-point (mean of log10 HIV RNA at 2 and 3 months after HIV+ diagnosis) Note: lowering of viral load set-point will presumably prevent or delay the onset of AIDS 3 POC Efficacy Trial (continued) • Null Hypothesis: Vaccine is same as Placebo VE = 0 and = 0 • Alternative Hypothesis: Vaccine is better than Placebo VE > 0 and/or > 0 true HIV infection rate for VACCINE VE = 1 true HIV infection rate for PLACEBO = true difference in mean viral load set- point among infected subjects [placebo – vaccine] • Proof-of-concept (POC) is established if the composite null hypothesis is rejected at = 5% 4 POC Efficacy Trial: Data Set-Up Number randomized Number HIV infected Proportion infected Viral load set-points of infected subjects (log10 copies/ml) Vaccine Placebo Nv nv nv Nv Np y1( v ) yn( vv ) y1( p ) yn( p ) p np np Np 5 Establishing POC: BOI vs. Simes’ Method Burden-of-Illness (BOI) (Chang, Guess, Heyse, 1994) nv Difference in BOI per subject: T Let Z BOI T E T | nv n p , H 0 Var (T | nv n p , H 0 ) y i 1 np (v) i Nv ( p) y i i 1 Np (see CGH) [unconditional test] Reject null if p value Pr Z Z BOI | H 0 .05 Simes’ method (Simes, 1986) p1 p-value for infection endpoint [unconditional test] p2 p-value for viral load endpoint [conditional test] Reject null if p value min(max( p1 , p2 ), 2 min( p1 , p2 )) .05 (Similar results with Fisher’s combined p-value method) 6 Power (%) to Reject the Composite Null Hypothesis (assuming = 1 log10 copies/ml) VE = 0% VE = 30% 100 80 80 60 60 40 40 20 20 0 0 Power (%) 100 10 30 50 70 90 10 30 50 70 90 VE = 60% 100 Power (%) 80 BOI Simes' 60 40 20 0 *Viral load reduction is 1 log10 in all plots 10 30 50 70 Number of HIV Infections 90 7 Adjusting for Selection Bias • Simulations led us to choose Simes’ over BOI for the POC efficacy trial. However … • Test for viral load component in Simes’ method: - Is restricted to subjects that are selected based on a post-randomization outcome (HIV infection) can suffer from selection bias. - Assesses mixture of (i) causal effect of vaccine and (ii) effect of variables correlated with VL that are unevenly distributed among the infected subgroups. References Rubin (1978), Rosenbaum (1984), Robins and Greenland (1992), Frangakis and Rubin (2002) 8 Adjusting for Selection Bias (continued) • Proposed approach - Adjust the viral load test for plausible levels of selection bias such that rejection of the null hypothesis becomes harder. - If the adjusted test is significant, then we have robust evidence of a causal vaccine effect. Hudgens, Hoering, Self (Statistics in Medicine, 2003) Gilbert, Bosch, Hudgens (Biometrics, 2003) [GBH] Mehrotra, Li, Gilbert (Biometrics, 2006) • Adjustment is derived via the principal stratification framework of causal inference (Frangakis and Rubin, Biometrics, 2002) 9 Adjusting for Selection Bias (continued) • Subjects infected under placebo {Si(p)=1} partition into the protected and always-infected principal strata Principal Stratum Potential infection outcome under Z Potential VL outcome under Z given Si(z) = 1 Protected Si(v) = 0, Si(p) = 1 undefined Yi(p,prot.) Always infected Si(v) = 1, Si(p) = 1 Yi(v) Yi(p,alw.inf.) Z = assigned treatment To assess a causal vaccine effect: we need to compare Yi(v) (= Yi(v, alw.inf.)) with Yi(p, alw.inf.), but the placebo VLs are a mixture of Yi(p, prot.) and Yi(p, alw. inf.) How to identify the distribution of Yi(p, alw.inf.)? 10 Adjusting for Selection Bias (continued) • fp(y) = (VE)fp(prot)(y) + (1-VE)fp(alw.inf)(y) fp(alw.inf)(y) = [w(y)/(1-VE)]fp(y) where w(y) = Pr{Si(v)=1|Yi(p)=y, Si(p)=1} is the unknown probability that a placebo infectee with VL set-point y would have been infected if given vaccine. • VE and fp(y) can be estimated from the data, but not w(y). Solution: assume a “known” model for w(y). 11 Adjusting for Selection Bias (continued) • GBH (2003) assume a logistic model for w(y): wi, = w(yi|,) = exp( + yi)/{1+exp( + yi)}, inp where is a fixed (pre-set) parameter: (i) = 0 wi, = 1 – VE for all i (ii) < 0 for a 1-unit decrease in Yi(p), the odds of being in the always infected stratum increase multiplicatively by exp(-) (iii) is a constant satisfying Fp(|) = 1 • For a given , fp(alw.inf) can now be estimated. 12 VL Distributions for the Protected and Always Infected Principal Strata Implied by the Logistic Model for wi(y) 2 3 4 5 log10 VL 6 7 0.4 0.3 0.2 1-VE VE 0.0 VE 0.0 1-VE = - (e- = ) 0.1 0.2 probability density 0.3 0.4 1-VE 0.1 = -2 (e- = 7.4) 0.1 probability density 0.2 0.3 VE 0.0 probability density 0.4 =0 (e- = 1) 2 3 4 5 log10 VL 6 7 2 3 4 5 6 7 log10 VL = 0: vaccine does not selectively protect subjects same distribution for Yi(p, prot.) and Yi(p, prot.) < 0: vaccine selectively protects subjects with higher VLs selection bias leads to biased estimation of the causal effect that makes the vaccine look poorer than it is. 13 Adjusting for Selection Bias (continued) • Adjust the viral load test in Simes’ method: 1) Fix the selection bias parameter 0. 2) Adjust (reduce) all the VLs of placebo infectees: ( p )* i , y n p yi( p ) n p w yi | ˆ, yi( p ) ( p) yi i1 i1 n p np w y | , ˆ i1 i ˆ is non-parametric m.l.e. of 3) Let T = Wilcoxon rank sum statistic comparing (v) i i nv {y } with { yi( p )*}in p 14 Adjusting for Selection Bias (continued) • When VE = 0, T is the Wilcoxon rank sum statistic used for the unadjusted VL test. • The distribution of T is intractable, so the p-value for the adjusted VL test ( = p2, ) is obtained using a nonparametric bootstrap. • Adjusted Simes’ method: for the specified , reject the composite null hypothesis if max( p1 , p2, ) or min( p1 , p2, ) / 2 • Robust evidence of a causal vaccine effect on either the infection or VL endpoint: reject the composite null hypothesis using the adjusted Simes’ method for all plausible values of . 15 BOI vs. Simes’ Method: Power (%) Assuming = 1 log10 copies/ml VE = 0% VE = 30% 100 80 80 60 60 40 40 20 20 0 0 Power (%) 100 10 30 50 70 90 10 30 50 70 90 VE = 60% 100 Power (%) 80 BOI Unadjusted Simes' (Beta=0) Adjusted Simes' (Beta=-1) Adjusted Simes' (Beta=-2) Adjusted Simes' (Beta=-Inf) 60 40 20 0 *Viral load reduction is 1 log10 in all plots 10 30 50 70 Number of HIV Infections 90 16 Concluding Remarks • The selection bias-adjusted Simes’ method is more powerful than the BOI method, unless VE is “large” (unlikely for a CMI-based HIV vaccine). • 50 events will provide at least 80% power to establish POC provided: VE 60% or 0.75 c/ml: unadjusted Simes’ method VE 60% or 1.0 c/ml: adjusted Simes’ method. • An -spending interim analysis after 30 events is proposed (details omitted here). Estimated time between 30 and 50 events is 9-15 months. 17 REFERENCES 1. Chang MN, Guess HA, Heyse JF (1994). Reduction in the burden of illness: a new efficacy measure for prevention trials. Statistics in Medicine, 13, 1807-1814. 2. Fisher RA (1932). Statistical Methods for Research Workers. Oliver and Boyd, Edinburgh and London. 3. Frangakis CE, Rubin DB (2002). Principal Stratification in Causal Inference. Biometrics, 58, 21-29. 4. Gilbert PB, Bosch RJ, Hudgens MG (2003). Sensitivity analysis for the assessment of causal vaccine effects on viral load in HIV vaccine clinical trials. Biometrics, 59, 531541. 5. Hudgens MG, Hoering A, Self SG (2003). On the analysis of viral load endpoints in HIV vaccine trials. Statistics in Medicine, 22, 2281-2298. 6. Mehrotra DV, Li X, Gilbert PB. Dual-endpoint evaluation of vaccine efficacy: Application to a proof-of-concept clinical trial of a cell mediated immunity-based HIV vaccine. Biometrics, in press. 7. Robins JM, Greenland S (1992). Identifiability and exchangeability of direct and indirect effects. Epidemiology, 3, 143-155. 8. Rosenbaum PR (1984). The consequences of adjustment for a concomitant variable that has been affected by the treatment. The Journal of the Royal Statistical Society, Series A, 147, 656-666. 9. Rubin DB (1978). Bayesian inference for causal effects: the role of randomization. The Annals of Statistics, 6, 34-58. 18 APPENDIX • Arguments against a selection-bias adjustment: - POC (not phase III) trial: not essential to precisely characterize the vaccine effect. - VE (and hence selection bias) anticipated to be small. - If vaccine prevents infection only for less virulent strains, then selection bias is more likely to make placebo look better than vaccine when comparing VLs, so the unadjusted test is already conservative from a causal inference perspective! • Arguments for a selection-bias adjustment: - Will we really proceed to phase III without robust evidence of a causal vaccine effect? - To satisfy statisticians who are wary of any nonrandomized comparison. 19 Hypothetical Example Infected/Enrolled VL set-point (log10 copies/ml) Vaccine Group 22/750 2.26 3.98 2.55 4.02 2.57 4.17 2.68 4.17 2.82 4.44 3.13 4.69 3.17 4.83 3.41 3.45 3.64 3.65 3.74 3.82 3.92 3.95 Placebo Group 28/750 2.79 4.45 3.26 4.57 3.32 4.58 3.51 4.66 3.72 4.92 4.02 4.99 4.08 5.18 4.10 5.19 4.10 5.20 4.14 5.23 4.20 5.52 4.21 5.60 4.24 5.62 4.26 4.40 VEobs 1 22 / 750 / 28 / 750 21% , p1 = 0.240 (binomial test) obs 4.43 – 3.59 = 0.84 log10 c/ml, p2 = 0.0001 (rank-sum test) Simes’ p-value = 0.0002*, BOI p-value = 0.062 20 Hypothetical Example (continued) Vaccine Group 22/750 Infected/Enrolled VL set-point (log10 c/ml) 2.26 2.55 2.57 2.68 2.82 3.13 3.17 3.41 3.45 3.64 3.65 3.74 3.82 3.92 X 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3.95 3.98 4.02 4.17 4.17 4.44 4.69 4.83 Placebo Group 28/750 X 0 0 0 1 1 0 1 1 2.79 3.26 3.32 3.51 3.72 4.02 4.08 4.10 4.10 4.14 4.20 4.21 4.24 4.26 X 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4.40 4.45 4.57 4.58 4.66 4.92 4.99 5.18 5.19 5.20 5.23 5.52 5.60 5.62 X 1 1 0 1 0 1 1 0 1 1 1 1 1 1 X = unobserved covariate (e.g., a genetic trait) A higher proportion of placebo infectees have X=1, and subjects with X = 1 tend to have higher viral loads. Did vaccine cause lower VLs or is the observed vaccine effect an artifact of the imbalance in the X distribution? 21 Hypothetical Example Revisited Assigning weights to the VLs in the Placebo Group log10 VL 2.79 3.26 3.32 3.51 3.72 4.02 4.08 4.10 4.10 4.14 4.20 4.21 4.24 4.26 0 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 weight (wi,) 2 0.99 0.98 0.98 0.97 0.96 0.93 0.92 0.92 0.92 0.92 0.91 0.90 0.90 0.89 1 1 1 1 1 1 1 1 1 1 1 1 1 1 log10 VL 4.40 4.45 4.57 4.58 4.66 4.92 4.99 5.18 5.19 5.20 5.23 5.52 5.60 5.62 0 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 weight (wi,) 2 0.86 0.85 0.82 0.82 0.79 0.69 0.66 0.57 0.57 0.56 0.55 0.41 0.37 0.36 1 1 1 1 1 1 1 1 0 0 0 0 0 0 22 Hypothetical Example Revisited 0.1 1-tailed p-value 0.075 Robust evidence of a causal vaccine effect on VL 0.05 0.025 = Inf 0 1 2 3 4 5 6 7 8 9 10 OR=exp 23