Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
SUBSTANTIAL EVIDENCE: Substantially Black and White By Moulakshi Roychowdhury, PharmD, JD 18 August 2011 Those of us who work in the regulatory advertising and promotions field often describe the regulations under which we operate and the US Food and Drug Administration (FDA) interpretation of those regulations as “gray.” We continuously scour FDA enforcement websites hoping to gain an inkling of what FDA’s current thinking is on a wide range of advertising and promotion topics such as risk presentation and social media. We attend conferences and hang on each regulator’s every word looking to be enlightened and move from gray to black and white. One exception to this gray zone is the “substantial evidence” standard. A thorough evaluation of enforcement trends indicates FDA has been unwavering in its application of this standard to both drug approvals and drug promotion. Of the 14 letters issued by the Division of Drug Marketing, Advertising, and Communications (DDMAC) in the first five months of 2011, 10 (71%) cited the need for substantial evidence when purporting promotional product claims. Similarly, of the 52 letters issued by DDMAC in 2010, 42 (81%) cited lack of substantial evidence. In each of these letters, DDMAC has qualified these citations with statements such as: ”Promotional materials are misleading if they represent or suggest that a drug is more effective than has been demonstrated by substantial evidence.”1,2 By sending these letters, DDMAC sent a loud and clear message it sees no flexibility in promotion with regard to the requirement for substantial evidence. But what does substantial evidence mean and how it is applied? Applicable Regulations With new drug applications (NDA), the Federal Food, Drug, and Cosmetic Act, 21 U.S.C. §355(d) defines “substantial evidence” as: “Evidence consisting of adequate and wellcontrolled investigations, including clinical investigations, by experts qualified by scientific training and experience to evaluate the effectiveness of the drug involved...“ Using this definition, FDA extends the substantial evidence standard specifically to advertising in 21 Code of Federal Regulations, 21 (CFR) 202.1 (e)(3)(ii)(b) as follows: “For which there exists substantial evidence of safety and effectiveness, consisting of adequate and well-controlled investigations… on the basis of which it can fairly and responsibly be concluded by such experts that the drug is safe and effective for such uses.” This is further elaborated in 21 CFR 314.126, which provides a comprehensive list of criteria for adequate and well-controlled investigations for a New Drug Application (NDA). Accordingly, when determining whether a study will provide substantial evidence for either drug approval or promotion, the following criteria must be satisfied by the study and study design: • The study must provide a clear statement of objectives. • A valid control group must exist to demonstrate a drug effect independent of other influences (e.g., placebo-controlled). • The study must select a patient population with a specific condition or which is at-risk of disease. • The baseline should be comparable (generally accomplished through randomization). • Steps must be taken to minimize bias (e.g., blinding). • There must be sufficient detail to allow for appropriate assessment and analysis of whether the data truly constitute evidence. Substantial evidence requires at least two adequate and well-controlled studies, each convincing on its own, to establish effectiveness. Occasionally, data from one adequate and well-controlled investigation with additional confirmatory evidence may be sufficient if FDA determines that this establishes effectiveness.3 As per FDA guidance, “Reliance on only a single study will generally be limited to situations in which a trial has demonstrated a clinically meaningful effect on mortality, irreversible morbidity, or prevention of a disease with potentially serious outcome and confirmation of the result in a second trial would be practically or ethically impossible.” A claim supported by two strong studies will always be considered more favorably and be more likely to constitute substantial evidence.4 Substantial Evidence Pitfalls While the above regulations are very clear, if we do not understand their application, the question remains whether DDMAC’s application of the “substantial evidence” standard in promotion is just as black and white. An analysis of DDMAC’s enforcement letters over the years, and particularly in the last few years, suggests that DDMAC consistently and rigidly applies this standard. A review of the enforcement letters identified a number of areas that were repeatedly deemed to not constitute substantial evidence: post-hoc subgroup analyses, meta-analyses, open-label studies, secondary endpoints and inconsistency with pre-specified statistical analysis plan (SAP) and endpoints. Regulatory Focus 19 Post-hoc Subgroup Analyses Currently, use of post-hoc analyses to make claims appears to be a red-flag for DDMAC. Post-hoc analyses generally do not have clearly defined endpoints with pre-specified statistical analysis plans (SAP), and are observational in nature. Consideration of countless characteristics or factors can be considered, causing significant tests to become difficult to interpret. This study design allows for confounding variables and bias and is generally unreliable. 5 These issues were highlighted in a DDMAC enforcement letter issued to a company for a professional journal advertisement. While the studies cited were “randomized, double-blind, and prospective” studies, the claims stemmed from the resulting data being retrospectively analyzed. Specifically, the claim came from a post-hoc subgroup analysis. DDMAC stated, “P-values are meaningless in this situation and cannot be used to demonstrate statistically significant differences between treatment groups … When looking for differences between treatment groups, the study must be designed to look for these differences prospectively, and must be sufficiently powered.”6 Meta-analyses A perception of bias has led to a general distrust of claims stemming from meta-analyses conducted by pharmaceutical companies. Meta-analyses combine multiple studies and summarize the results. They also increase the sample size and the power to study effects of interest. The caveats include selection bias, as unfavorable studies can potentially be omitted and patient populations may vary vastly from study to study. Essentially, meta-analyses are not well-controlled studies and have on occasion produced results inconsistent with those of large, well-controlled trials.7 DDMAC is mindful of these caveats and has shared these same concerns.8 For example, when a certain company used a meta-analysis of 43 double-blind studies to claim efficacy in a sales aid, DDAMC stated in an enforcement letter that the claim was not supported by substantial evidence. In this letter, DDMAC reiterated, “A claim of superiority should be based on a comparison of the maximum dose of two products in two adequate, well-designed, head-to-head clinical trials.”9 In addition, meta-analyses of “across label” comparisons are not considered to be substantial evidence. DDMAC issued an enforcement letter to a company that utilized a meta-analysis of 18 different product labels comparing relative abuse liability to support a claim of superiority. This was a real problem, as the FDA-approved indications for the products being compared were not the same. DDMAC noted that a disclaimer was included in the piece clarifying that the claim stemmed from a meta-analysis and 20 August 2011 not a head-to-head clinical trial, but said that the disclaimer did not “mitigate the misleading implication.”10 Open-label Studies Open-label studies are those in which both the patients and investigators are aware of which treatment is being administered, and they are especially problematic when the endpoints being evaluated are based on subjective patient or investigator responses, as this allows for biases. This concern about subjective responses was underscored in a DDMAC enforcement letter issued for a journal advertisement which touted a claim supported by a “randomized, open-label, multi-center study of 3097 subjects with two treatment arms.” Although the study was randomized, it still allowed for bias because the endpoint being evaluated (depression) was subjective. DDMAC further opined, “Blinding is intended to minimize the potential biases resulting from differences in management, treatment, or assessment of patients, or interpretation of results that could arise as a result of subject or investigator knowledge of the assigned treatment … while nominally statistically significant in this very large study (over 3,000 patients), cannot be relied upon as substantial evidence.”11 Similarly, another company was cited for a visual aid that made efficacy claims based on the results of an open-label, uncontrolled clinical study. DDMAC explained, “Results from a single open-label clinical trial with no control group do not constitute substantial evidence… to support this, or any other, efficacy claim.”12 Secondary Endpoints Secondary endpoints are tricky, as they are often dependent on the primary endpoints. Generally, if a study misses its primary endpoint, this renders the secondary endpoint useless. Also, if a study’s primary endpoint is met, then the secondary endpoints must be pre-specified and statistically designed to meet significance. Elaine Hu, of DDMAC, clarified, “Generally, only primary and key secondary endpoints can support claims; primary and key secondary endpoints are endpoints that have a pre-specified analysis plan with explicit allocation of alpha (the possibility of making a false assumption).”13 DDMAC issued an enforcement letter to a company for a professional reprint carrier that claimed superiority to another drug based on a secondary endpoint of an open-label, randomized study because the study “failed to demonstrate statistical significance on the primary endpoint and has not been replicated.” DDMAC explained that, “In the absence of a finding of statistical significance for the primary endpoint, any further analysis conducted on this study, including secondary endpoints and subpopulations, are all considered exploratory and do not generally constitute substantial evidence.”14 Similarly, DDMAC issued another letter for a product website that included efficacy claims based on a secondary endpoint. However, the study cited only achieved significance on one of the two primary endpoints, rendering the results of the secondary endpoint invalid. DDMAC explained that “Generally, claims based on secondary endpoints are problematic when, as is the case here, the study fails to meet statistical significance on all primary endpoints … in the absence of a statistically significant improvement in overall response rate AND time to (primary endpoint), it is misleading to make claims based on secondary endpoints from the clinical study.”15 Inconsistency with Pre-specified Statistical Analysis Plan (SAP) and Endpoints Essentially, through enforcement, DDMAC has made it clear that it is not sufficient that a claim stems from the results of a well-controlled study, but that the claim being made must be consistent with what the study is designed to measure. For instance, if a study is prospectively and statistically designed to measure a difference between placebo and treatment at four weeks, then claiming a benefit at two weeks is inappropriate. This was illustrated in a DDMAC enforcement letter issued for a direct-to-consumer (DTC) patient brochure that claimed effects of a drug on skin at specific time points (specifically, two weeks and one month). However, the primary efficacy endpoint was designed to measure a difference at week 10. The other time points were not pre-specified and were analyzed retrospectively. That made the shorter time points invalid, and they were considered to not constitute substantial evidence.16 Very recently, DDMAC issued an enforcement letter to a company for a detail aid that contained a number of comparative claims that suggested overall superiority over time. Each of these claims stemmed from two studies. While they were well-controlled, the studies focused on one specific time point of comparison. However, the products were each respectively approved for symptom relief over an extended period of time, and not solely one time period. Therefore, superiority over one time point would not constitute substantial evidence to support a claim of overall superiority. DDMAC ruled that the piece was misleading as it suggested that the drug was “more effective than another drug, when this has not been demonstrated by substantial evidence.”17 Real-world Navigation of the Substantial Evidence Standard: Advice for Industry Given that the application of the substantial evidence standard may be black and white, but with a number of pitfalls, ensuring that a claim is supported by substantial evidence may require a step-by-step approach. Provided below are suggested questions to ask when navigating this analysis: 1. Does the proposed claim stem from the data provided in the FDA-approved label? If yes, there may be substantial evidence. 2. If not, is the proposed claim consistent with the indication and related clinical outcomes? If no, end analysis here. If yes, continue on to question 3. 3. If yes, is there substantial evidence to support the claim (generally, exclude post-hoc subgroup analyses, metaanalyses, open-label studies and secondary endpoints with failed primary endpoints)? 4. If making a comparative claim, are there two adequate and well-controlled studies supporting the claim? 5. If making a single product claim, are there two adequate and well-controlled studies supporting the claim? If there is only one study, is there strong confirmatory evidence? 22 August 2011 Those of us who work on promotional review committees soon learn that people who do not fervently comb through DDMAC enforcement letters on a daily basis are not always aware of the paramount need for substantial evidence to support promotional claims. Without this knowledge, frequently, by the time pieces are routed for promotional review, a great deal of expense, time and energy has already been invested in developing claims that may not be appropriate for promotion. To avoid wasting these resources, it may be helpful to train those involved in the creation of these pieces ahead of time so they are aware of how to avoid pitfalls. Furthermore, which professionals are typically the most well-versed in study designs and the clinical validity of study results? The prime candidates are those who specialize in medical affairs. To help companies avoid enforcement due to lack of substantial evidence, medical colleagues should be actively engaged in the review of promotional materials. In evaluating for substantial evidence, medical may be able to determine whether endpoints measured are well-defined and reliable, whether the data are meaningful, and whether they can be interpreted without bias to reach a specific claim, to name a few. In essence, medical may be very valuable in the critical evaluation and determination of whether the characteristics of an adequate and well-controlled study are present as provided in 21 CFR 314.126. Further, these experts should also review DDMAC enforcement letters to be aware of the distinction between the “substantial evidence” standard as applied by DDMAC versus what may be appropriate in clinical practice. Lastly, it may be beneficial for regulatory professionals responsible for promotional review and marketing to be involved with regulatory affairs and development during the early stages of designing studies to provide guidance on study designs and set realistic expectations with commercial organizations. In addition, regulatory personnel should have clear discussions with commercial designers early on. They should talk about what information is important to communicate to physicians and/or consumers in a particular disease state and ensure that studies (if possible) are designed adequately to measure these endpoints, and that there exists a pre-specified SAP. In addition, DDMAC suggests that sponsors discuss these desired claims with the appropriate review division to determine how to best plan for the interpretation of study findings to avoid conducting studies that are not able to support the sponsors’ desired claims.18 Stay Within the Black and White Lines In summary, the substantial evidence standard is one area where DDMAC is unwavering. The countless DDMAC enforcement letters underscore the necessity for substantial evidence when making promotional claims, and it is our responsibility as regulatory professionals to communicate this message and educate our industry colleagues. References 1. Patel S. DDMAC Untitled Letter to Dow Pharmaceutical Sciences Inc. for Acanya. Jun 16 2011. 2. Panholzer L. DDMAC Untitled Letter to Allergan Inc. for Acuvail. 25 August 2010. 3. Hu Cunningham E. Levels of Evidence to Support Promotional Claims for Prescription Drugs. Presentation at Drug Information Association Meeting. 23 February 2011. 4. Guidance for Industry: Providing Clinical Evidence of Effectiveness for Human Drug and Biological Products. May 1998. 5. Sleight P. Debate: Subgroup analyses in clinical trials: fun to look at- but don’t believe them! Current Control Trials in Cardiovascular Medicine. 2000 1(1): 25-27 6. Abrams T. DDMAC Warning Letter to Pfizer Inc. for Zyvox. 20 July 2005. 7. Walker E, et al. “Meta-Analysis: Its Strengths and Limitations.” Cleveland Clinical Journal of Medicine. 75(6). 431-439. Jun 2008. 8. Op cit 30. 9. Abrams T. DDMAC Warning Letter to Sankyo Pharma for Benicar. 6 January 2006. 10. Patel S, et al. DDMAC Untitled Letter to Takeda Pharmaceuticals North America Inc. for Rozerem. 28 January 2010. 11. Abrams T. DDMAC Warning Letter to Wyeth Pharmaceuticals Inc. for Effexor XR. 10 December 2007. 12. Abrams T. DDMAC Warning Letter to Galderma Laboratories for Tri-Luma Cream. 18 August 2009. 13. Op cit 30. 14. Olin K. DDMAC Untitled letter to Sanofi Aventis for Taxotere. 16 April 2009. 15. Abrams T. DDMAC Warning Letter to Eisai Inc. for Dacogen. 6 November 2009. 16. Patel S. DDMAC Untitled Letter to Sanofi Aventis for BenzaClin. 22 January 2010. 17. Op cit 1. 18. Op cit 30. Author Moulakshi (Mou) Roychowdhury, PharmD, JD, currently works at Biogen Idec as Associate Director, Regulatory Affairs - Advertising, Labeling, and Promotion. She is a graduate of the Ernest Mario School of Pharmacy, Rutgers University and Brooklyn Law School. Roychowdhury has worked in similar capacities at Bayer HealthCare Pharmaceuticals and Genzyme Corporation advising the companies on promotional compliance with FDA regulations. Regulatory Focus 23