Download Scientific and Ethical Issues in Equivalence Trials

Editorials represent the opinions
of the authors and THE JOURNAL and not those of
the American Medical Association.
EDITORIAL
Scientific and Ethical Issues
in Equivalence Trials
Benjamin Djulbegovic, MD, PhD
Mike Clarke, DPhil
A
NY TESTING OF MEDICAL TREATMENTS IS AN EXERcise in comparison. In a typical clinical trial, 2 treatments are compared to determine which is better
or if both are the same. Trials designed to address whether one treatment is better than the other may
be called superiority trials, whereas those designed to show
that 2 treatments are the same are called equivalence trials.
However, the design of both types of trials should depend
on the uncertainty principle—a fundamental ethical and
scientific principle for conducting randomized controlled
trials.1
The article by Staszewski et al2 in this issue of THE
JOURNAL is a randomized controlled equivalence trial that
compares a triple nucleoside regimen of abacavir-lamivudinezidovudine with a more conventional regimen of indinavirlamivudine-zidovudine in treatment-naive patients infected with human immunodeficiency virus (HIV). Although
the authors conclude that these 2 regimens are equivalent
in achieving the primary end point of reducing plasma HIV
RNA levels to below 400 copies/mL, several factors make
the interpretation of this study and other equivalence trials
particularly difficult.
In planning a clinical trial of a new intervention, 2 main
issues must be addressed. The first is the fundamental ethical question of whether the use of the new intervention is
justified. The second is the choice of the appropriate control group.3 Both issues are fundamentally related to the preexisting knowledge about the therapeutic value of the treatments to be compared. This is an important reason that
clinical trials should be preceded by a systematic review to
assess the status of this knowledge, and should be reported
with a discussion of an updated review including the trial’s
results.4 The trial would not be justified if one of the treatments to be assessed is known to be superior to the other.
A clinical trial is only justified if the patient and clinician
are not certain about which treatment to choose from the
available options. If they are uncertain (indifferent) about
the relative value of the treatments, it is time for a trial.5 This
is not only because the trial will help resolve this uncer-
See also p 1155.
1206 JAMA, March 7, 2001—Vol 285, No. 9 (Reprinted)
tainty but also because it is the fairest way to choose the treatment for the patient. Patients enrolled in the study have a
50% chance of receiving the better treatment and the overwhelming weight of evidence is that they will fare better while
participating in the trial (regardless of the treatment they
are allocated to) than while outside of it.6 This realization
forms a basis for the scientific and ethical underpinnings
for the design and conduct of randomized trials, expressed
in the term uncertainty principle, which states that a patient
should be enrolled in a randomized controlled trial only if
there is substantial uncertainty about which of the trial treatments would benefit the patient more.7
Basing trials on the uncertainty principle also addresses
another important issue in the design of a clinical trial—
the choice of an adequate comparator for the intervention
under investigation.8 Studies in which the intervention and
the control or comparison group are known in advance to
be nonequivalent in their effects on the main outcomes of
interest violate the uncertainty principle.8 Even if a study
is properly reported,9 extra caution might be needed in its
interpretation if the choice of the comparison treatment was
not based on uncertainty about the relative value of the treatments being assessed.8,10
Uncertainty can have many grades ranging from simply
not knowing11 to maximum uncertainty (also known as equipoise)11,12 about the relative benefits and harms of the treatment alternatives. The uncertainty might be in the mind of
the patient, the clinician, or the community.12
Most clinical trials are assessments of superiority and start
with the statement of a null hypothesis of no difference between 2 therapies. That is, prior research should not have
proved a difference between the alternative treatments in
the outcomes to be assessed. The trial is designed to reject
the null hypothesis by showing that there is a difference between the treatments. Since the null hypothesis can never
be proven, but only rejected,13 alternative hypotheses (ie,
that one treatment is better) are not assessed directly, but
are accepted if the probability that the observed results are
Author Affiliations: Interdisciplinary Oncology Program, Division of Blood and Bone
Marrow Transplantation, H. Lee Moffitt Cancer Center and Research Institute, University of South Florida, Tampa (Dr Djulbegovic); and Cochrane Centre, NHS Research and Development Program, Summertown Pavilion, Oxford, England (Dr
Clarke).
Corresponding Author and Reprints: Benjamin Djulbegovic, MD, PhD, H. Lee Moffitt Cancer Center and Research Institute, University of South Florida, 12902 Magnolia Dr, Tampa, FL 33612 (e-mail: [email protected]).
©2001 American Medical Association. All rights reserved.
EDITORIALS
obtained by chance is less than some predetermined level
of statistical significance.14
Ethical principles expressed in terms of explicit acknowledgment of uncertainty seem to reflect a conventional trial design. For example, the US Department of Health and Human
Services’ Office for Human Research Protections Institutional
Review Board Guidebook15 states that before beginning a randomized controlled trial, researchers should honestly be able
to state a null hypothesis reflecting their uncertainties “that
subjects treated with . . . the trial therapy will not differ in outcome from subjects treated with . . . the control therapy.” The
trial must be designed in such a way that “its successful completion will show which of the therapies is superior.”15 Equivalence trials, on the other hand, set out to prove that treatments are not different. The null hypothesis to be tested and
disproved if the trial is to be successful by showing equivalence is actually that the treatments are different.16,17
The process of a clinical trial design starts with acknowledgment and definition of uncertainties around relative values of treatments to be tested, conversion of these uncertainties into a null hypothesis, and its translation into informed
consent. Currently, informed consent for superiority and
equivalence trials usually do not differ. However, the uncertainties, purpose, and design of these trials are, in fact, mirror
images of each other. Therefore, the informed consent for participation in clinical trials should reflect this difference.
The consent process should first acknowledge uncertainties with a statement such as, “it is unknown which drug is
better or even if they are the same. The purpose of this trial
is to help find out.” In a superiority trial, informed consent
might also mention that “the new drug might be better than
the old drug, but it might be the same or even worse. To
prove it, scientists start with the assumption that there is
no difference between the two drugs.” On the other hand,
the basic premise in an equivalence trial might be supplemented by a statement such as “this trial is designed on the
basis of current beliefs that the new drug might be no different than the old drug. However, to prove it scientists need
to start with the assumption that the new drug is better.”
This is the fundamental ethical challenge of equivalence trials. It is commonly believed that human experimentation involves an unavoidable “tension between conduct of a trial and
the autonomy of the individual” and that patients are asked
to make a sacrifice for the good of others, particularly when
it comes to the use of placebo.18,19 However, as long as there
is substantial uncertainty about which treatment is superior,
patients do not lose out prospectively and are not required to
subjugate their interests and well-being for the benefit of others.1 However, in equivalence trials, the hypothesis to be tested
(and, therefore, refuted if equivalence is to be shown) is that
one treatment is superior. If the trial is described in this way,
researchers might expect that only altruistic patients would
be willing to be enrolled in equivalence trials because other
patients may want to request the treatment that investigators
assumed is better in their prior hypothesis.
©2001 American Medical Association. All rights reserved.
Several factors may make the interpretation of equivalence
trial results particularly difficult. When designing a superiority trial, a power calculation and sample size determination
are performed to assess the probability that a given difference is obtained by chance. In equivalence trials, this difference ideally would be zero, although a proof of exact equality
is not possible.16 In practice, this issue is resolved by defining
an arbitrary practical equivalence margin, also called the noninferiority margin.17 To detect this difference, on average,
equivalence trials usually will require a 10% larger sample size
in comparison with conventional superiority trials.16 The null
hypothesis would be rejected if the upper limit of the confidence interval for the difference between the treatments is
smaller than this predefined margin.17 Setting of the margin
is critical and should be chosen on the basis of excluding a
clinically important difference between the treatments. However, the definition of what constitutes such a difference may
vary widely for each patient and clinician, and might fall below the margins set by the trialist.16,17
In this study, Staszewski et al2 set the limit for the difference at 12% for their primary end point (a plasma HIV RNA
level of 400 copies/mL at week 48) based on discussions
among clinical investigators of the study and with officials
from the Food and Drug Administration.
Once the study results are obtained, a key question for
the interpretation of equivalence trials revolves around
whether both treatments were effective, or whether the result indicates that both treatments were ineffective.17 This
could also be a feature in the interpretation of superiority
trials and is one reason that a placebo or no treatment control arm would be used (if an active control treatment does
not exist).17,20 Of course, a substantial amount of historical
data indicate that the lack of a treatment arm would not be
appropriate for studies such as the trial by the Staszewski
et al involving patients with HIV infection who require treatment. Nonetheless, its interpretation remains problematic.
Absence of evidence (of a difference) must not be confused
with evidence of absence (of a difference).21 The observation of a lack of a difference between 2 treatments cannot
automatically be used as evidence of equivalence.16
An additional problem is that common techniques used to
minimize bias in clinical trials are less useful in equivalence
trials. In conventional trials, use of randomization, blinding,
and intent-to-treat analysis serve 1 purpose: to ensure comparability between the 2 groups in all respects other than the
study treatment so that any outcomes that differ between the
groups could only be due to the study treatment or to chance.
However, when the intent is to show that the study treatment is identical to control, techniques that ensure similarities between 2 groups are less helpful.3,16,17 Indeed, in the study
by Staszewski et al, results of the intent-to-treat analysis differ from the as-treated analysis, with the more conventional
regimen of indinavir-lamivudine-zidovudine actually appearing to do better.2 Although reasons for this difference are difficult to discern without examination of the actual data in the
(Reprinted) JAMA, March 7, 2001—Vol 285, No. 9
1207
EDITORIALS
trial, there were large differences in the proportions of patients available for the intent-to-treat and as-treated analyses
in the 2 treatment groups. For example, for the primary end
point, the difference in proportions of patients analyzed was
55% (133/262 vs 125/145 in the abacavir-lamivudinezidovudine group) and 52% (136/265 vs 130/139 in the indinavir-lamivudine-zidovudine group). Therefore, the apparent equivalence reflected in the intent-to-treat analysis might
simply be due to a dilutional effect of comparing 2 groups of
patients whose actual treatments did not differ much.
The design of a clinical trial should be a function of the uncertainty principle, which should underpin both superiority
trials and equivalence trials. However, the former are usually
designed in the hope that one treatment will prove better than
the other, whereas the latter are designed in the hope that both
are the same. The main impetus for an equivalence trial is the
notion that proving equal efficacy may enable patients to have
treatments that are not more effective than existing ones, but
are better for some other reason.20 However, if 2 treatments
are shown to have equal efficacy, the evidence that one is better should also be of the highest standard possible. Clinicians
should be cautious about arguing for equivalence based on a
randomized trial of efficacy, and then using arguments about
toxicity (or some other end point that was not formally assessed in the trial) as a basis for suggesting the superiority of
one of the treatments. Problems identified with the interpretation of equivalence trials should not necessarily argue against
their conduct. Rather, such trials need to be designed and reported in a transparent and explicit fashion, to acknowledge
that they are not really equivalent to superiority trials.
REFERENCES
1. Edwards SJL, Lilford RJ, Braunholtz DA, Jackson JC, Hewison J, Thornton J. Ethical issues in the design and conduct of randomized controlled trials. Health Technol Assess. 1998;2:1-130.
1208 JAMA, March 7, 2001—Vol 285, No. 9 (Reprinted)
2. Staszewski S, Keiser P, Montaner J, et al. Abacavir-lamuvidine-zidovudine vs
indinavir-lamuvidine-zidovudine in antiretroviral-naive HIV-infected adults: a randomized equivalence trial. JAMA. 2001;285:1155-1163.
3. International Conference on Harmonization: choice of control group in clinical
trials, 64 Federal Register 51767 (1999).
4. Clarke M, Chalmers I. Discussion sections in reports of controlled trials published in five general medical journals: islands in search of continents? JAMA. 1998;
280:280-282.
5. Bradford Hill A. Medical ethics and controlled trials. BMJ. 1963;2:1043-1049.
6. How do the outcomes of patients treated within randomised control trials compare with those of similar patients treated outside these trials? Available at: http:
//hiru.mcmaster.ca/ebm/trout/. Accessibility verified February 1, 2001.
7. Peto R, Baigent C. Trials: the next 50 years. BMJ. 1998;317:1170-1171.
8. Djulbegovic B, Lacevic M, Cantor A, et al. The uncertainty principle and industrysponsored research. Lancet. 2000;356:635-638.
9. Begg CB, Cho MD, Eastwood S, et al. Improving the quality of reporting of randomized controlled trials: the CONSORT statement. JAMA. 1996;276:637-639.
10. Djulbegovic B, Bennett C, Lyman G. Violation of the uncertainty principle in
conduct of randomized controlled trials (RCTs) of erythropoietin (EPO). Blood. 1999;
94:399A.
11. Lilford RJ, Jackson J. Equipoise and the ethics of randomization. J R Soc Med.
1995;88:552-559.
12. Lilford RJ, Djulbegovic B. Equipoise and “the uncertainty principle” are not
two mutually exclusive concepts. [An electronic response to: Clinical equipoise and
the uncertainty principle is the moral underpinning of the randomised controlled
trial. BMJ. 2000;321:756-758.] Available at: http://www.bmj.com. Accessibility
verified February 12, 2001.
13. Popper K. The Logic of Scientific Discovery. New York, NY: Harper & Row;
1959.
14. Hulley SB, Cummings SR. Designing Clinical Research. Baltimore, Md: Williams & Wilkins; 1992.
15. Office for Human Research Protections. Institutional Review Board Guidebook. Rockville, Md: US Dept of Health and Human Services; 1993. Available at:
http://ohrp.osophs.dhhs.gov/irb/irb_guidebook.htm. Accessibility verified February 1, 2001.
16. Senn S. Statistical Issues in Drug Development. New York, NY: John Wiley &
Sons Inc; 1997.
17. Temple R, Ellenberg SS. Placebo-controlled trials and active-control trials in
the evaluation of new treatments, part 1: ethical and scientific issues. Ann Intern
Med. 2000;133:455-463.
18. Rhotman KJ, Michels KB, Baum M. Declaration of Helsinki should be strengthened: for and against. BMJ. 2000;321:442-445.
19. Mathe G, Brienza S. From methodology to ethics and from ethics to methodology. Biomed Pharmacother. 1988;42:143-153.
20. Ellenberg SS, Temple R. Placebo-controlled trials and active-control trials in
the evaluation of new treatments, part 2: practical issues and specific cases. Ann
Intern Med. 2000;133:474-475.
21. Altman D, Bland M. Absence of evidence is not evidence of absence. BMJ.
1995;311:485.
©2001 American Medical Association. All rights reserved.