Download Issues in Surgical Randomized Controlled Trials

World J. Surg. 23, 1210 –1214, 1999
WORLD
Journal of
SURGERY
© 1999 by the Société
Internationale de Chirurgie
Issues in Surgical Randomized Controlled Trials
Robin S. McLeod, M.D.
Departments of Surgery and Public Health Sciences, Division of General Surgery, University of Toronto, Mount Sinai Hospital, 600 University
Avenue, Room 449, Toronto, Ontario M5G 1X5, Canada
Abstract. As technology expands and health care resources become more
limited, there is increasing pressure on surgeons to evaluate surgical
procedures and technology to ensure they are effective. No longer is
evidence from case series acceptable, and treatments must be evaluated in
randomized controlled trials. The power of the randomized controlled
trial is that because of its rigorous design the risk of random or systematic (bias) error is minimized and thus the risk of making an incorrect
conclusion about the efficacy of a treatment is minimized. The randomized controlled trial has been widely accepted for evaluating the efficacy of
medical treatments but less so for surgical procedures. Many cite methodologic issues related to surgical issues as the reason for this, including
concerns related to standardization of the surgical procedure, timing of
surgical trials, difficulties with blinding of subjects and investigators,
ethics of surgical trials, and patient and surgeon acceptance of surgical
trials. These issues are discussed in this paper as are possible strategies
to minimize their effect.
otic therapy alone [1], trials comparing various antibiotic regimens
for treatment of intraabdominal sepsis [2], and new drug therapies
being tested in patients with acute ulcerative colitis where previously surgery was accepted as the only alternative [3].
For the investigator, the challenge when performing an RCT is
ensuring that the two groups are truly the same and that protocol
deviations do not occur that could invalidate the results because
of random or systematic errors. Performing an RCT that is methodologically rigorous is difficult no matter what treatments are
being compared, and most clinical researchers agree that performing trials where one or two surgical therapies are being
evaluated are especially challenging. The focus of this paper is a
discussion of some of the problems encountered in surgical trials
and some of the possible solutions to them.
The randomized controlled trial (RCT) is considered the optimal
design for determining treatment effectiveness. The power of the
RCT is that subjects are randomly allocated to two (or sometimes
more) treatment groups and are followed in an identical manner
prospectively with the outcome of interest defined a priori. Thus
the groups are similar with respect to both known (or measured)
and unknown attributes except for the treatment they receive.
Therefore it can be assumed that if a difference is observed it is
due to a difference in the effectiveness of the two treatments.
Randomized controlled trials are a phenomenon of the latter
part of the twentieth century. Previously, most clinicians relied on
careful observation to make deductions about the efficacy of a
treatment. There are many examples of accepted therapies that
were introduced to clinical practice without being tested in an
RCT, including appendectomy, antibiotics for treating intraabdominal infections, and colectomy for ulcerative colitis. What
these interventions have in common is that the treatment effects
are so large, compared to the alternative or no treatment, that
their effectiveness is self-evident; RCTs are unnecessary to confirm these observations. More commonly, in modern medicine
technologic developments or surgical interventions lead to small
increments in survival or perhaps no difference in survival, only
improvement in quality of life. It is necessary, therefore, that
extraneous factors are controlled to be certain that the observed
difference is indeed due to the treatment. Even therapies that
were once accepted without question are being evaluated in
RCTs. Thus there are trials comparing appendectomy to antibi-
Standardization of the Surgical Procedure
In medical trials the dosage can be standardized, and compliance
can be measured. Furthermore, the prescribing physician is not a
variable. In surgical trials, however, standardization of a procedure may be difficult because surgeons vary in their experience
with, and their ability to perform, a surgical technique; there may
be individual preferences when performing the procedure; and
there may be technical modifications to the procedure as it
evolves. Furthermore, differences in peri- and postoperative care
may affect the outcome. Many view this as a major limitation when
performing surgical trials, but one can look at the issue from the
opposite viewpoint. In medical trials, there must be continuous
monitoring of patients to ensure compliance with taking medication for the duration of the trial. In surgical trials, once the
procedure is completed, “compliance” no longer becomes an
issue, monitoring is unnecessary, and strategies to ensure compliance are not needed.
Standardization is an important issue when designing a surgical
trial, and a certain minimum is mandatory. The amount of standardization may vary depending on the question being asked,
similarly to the fact that compliance may vary in a medical trial.
Thus a trial where surgical technique is highly controlled (i.e., few
surgeons who are experts in the field using a similar technique) is
analogous to the medical trial where only compliant patients are
randomized. This may be viewed as an efficacy trial. If the surgical
maneuver is less controlled, it may be viewed as an effectiveness
McLeod: Issues in Surgical RCTs
trial. In other words, the question is: “Is this surgical procedure
effective when performed by many different surgeons without
special expertise with this procedure?” Depending on the design
of the trial, the generalizability of the results varies. No matter
what strategy is adopted, it is essential that the investigator describe how the procedure was performed and by whom so the
reader can determine whether the results are applicable to his or
her practice and the technique can be replicated.
It is as important to standardize the technique of the control
operation as it is for the new operation. It is often easier to do so,
as the procedure may be more “mature” and the technique may
be more standardized. Other aspects of care deemed important,
such as prophylactic antibiotic usage and postoperative care,
should also be standardized.
Several strategies may be instituted to ensure that uniformity in
surgical technique is achieved and that the effect of the surgeon as
a variable is minimized. First, one can ensure that there is a
consensus among participating surgeons on how to perform at
least the critical aspects of the procedure. Prior to the trial,
teaching sessions may be held to improve standardization. Manuals may be produced that describe and illustrate the accepted
technique. Second, during the trial measures can be instituted
(e.g., pathologic examination of resection margins, radiographic
evidence of vessel patency) to audit the surgical performance. If
results are suboptimal, performance may be improved by providing feedback to surgeons. Third, patients may be stratified by the
surgeon. There may still be variation in how the procedure is
being performed, but stratification ensures that there is not an
imbalance between groups. Finally, participation may be limited
to a small number of surgeons or only those for whom there is
documentation that a satisfactory number of procedures have
been performed or there is evidence they can perform it adequately. The advantage of the latter strategy is that uniformity of
technique is more likely. It increases the likelihood of observing a
treatment effect, as only experts are included. The disadvantage is
that the results may be less generalizable.
Timing of Trials
Chalmers argued that the first patient on whom a surgical procedure is performed should be randomized [4]. However, his reasoning was mainly based on ethical concerns rather than methodologic ones. Most surgeons agree that there is a learning curve for
any procedure and that modifications to the technique are frequently made at its inception. By including these early patients,
one would almost certainly bias the results against the new procedure. It might also inhibit modifications being made as experience with the procedure increases. Alternately, if modifications
are not made, the procedure, as it is performed in the trial, might
be obsolete at the completion of the trial, and thus the results
might have little impact. Delaying the institution of a trial until the
procedure has been perfected also has its hazards. A trial may be
difficult to initiate when the procedure has become widely accepted not only by the surgical community but also by patients.
The paucity of RCTs testing surgical therapies may be due in large
part to this dilemma related to the timing of the trial. However,
the EC-IC Bypass and NACET studies are examples of trials that
were performed even though the procedures had already gained
wide acceptance [5, 6]. On the other hand, laparoscopic cholecystectomy has not been formally evaluated in any large trials [7, 8].
1211
Although this may be a significant failure to evaluate a new
surgical procedure adequately, it may be an example of where an
RCT is unnecessary because laparoscopic cholecystecomy so
clearly results in less pain and earlier discharge (the main outcome variables). Certainly, if trials had been performed early,
almost certainly they would have been biased against laparoscopic
cholecystecomy with respect to bile duct injuries.
There is therefore no optimal time to test a procedure in an
RCT. If at any time after its introduction there are clinicians who
are uncertain of the effectiveness of a procedure, a randomized
controlled trial should be performed. Investigators must accept
that this may not be the definitive trial because procedures are
continually being “fine tuned,” and other technical and pharmaceutical advances in the field may alter the indications for surgery.
Medical oncologists are always planning a new trial as they complete one based on the results of previous trials. This same attitude should be adopted by surgical investigators.
Blinding of Subjects and Investigators
Bias may occur owing to the lack of blinding of patients or
investigators (or both) in any clinical trial whether it is assessing
medical or surgical therapies. This is of great concern in surgical
trials, where it is known that surgery often has a significant
placebo effect [9]. In trials comparing medical and surgical therapies, blinding may be impossible because sham operations are
generally unethical. Even in trials comparing two surgical operations there may be difficulty blinding patients and investigators if
the incisions differ or the operations differ in magnitude. Creative
maneuvers may be required to minimize unblinding. As an example, in a trial comparing minicholecystectomy to laparoscopic
cholecystectomy, a large dressing covering the entire abdomen
was used to blind surgeons, nurses, and patients to the procedure
performed [8].
The lack of blinding may be minimized by choosing a “hard”
outcome measure, such as recurrence of disease or death. However, more often the important outcome is a change in symptoms
or quality of life, and outcome may be biased by knowing which
treatment group the subject is in. Because the quality of life
assessment made by the patient is of greatest importance, blinding
may be impossible. In these situations, the potential for bias may
be minimized if a hard outcome measure is also assessed and it
correlates with the patient’s subjective assessment. For example,
in the VA Cooperative Study Group on Gastroesophageal Reflux
Disease (GERD), patients’ assessments of their reflux symptoms
plus endoscopic findings were recorded [10]. If possible, assessments should be performed by an independent assessor who is
unaware of the treatment group the patient is in. For example, an
endoscopist or a radiologist should be “blinded” when they perform procedures or interpret results. Finally, by explicitly defining
outcome criteria a priori, bias may be minimized even if the
assessors are unblinded. A blinded panel may also be used to
review results of tests using these criteria but without knowledge
of the treatment allocation.
The importance of blinding may vary depending on the trial and
the primary outcome measure. For instance, if the outcome measure is all-cause mortality, even if there is no blinding the results
will probably not be biased. On the other hand, if the primary
outcome is quality of life, lack of blinding may potentially bias the
1212
results and the investigators should spend considerable effort to
minimize it.
Analyzing Results
For the most part, analysis of the results is the same irrespective
of the treatments being evaluated. Most statisticians agree that an
intention to treat analysis is the preferred method. Also known as
an effectiveness analysis, it is performed by including all randomized patients in the analysis. Patients are always analyzed in the
group to which they were randomized. Peto et al. argued that if
the failure to complete the trial or receive the correct treatment
occurred owing to chance there should be an equal proportion of
patients with protocol deviations in each group [11]. If so, by
including all patients in the analysis the treatment effect, if there
is one, may be reduced; otherwise, the results are not biased. On
the other hand, if the failure is due to an effect of the treatment
(e.g., toxicity or intolerance of the treatment), it is not due to
chance; and omission of those patients from the analysis might
bias the results in favor of one of the treatments. Finally, all events
that occur after randomization must be included in the analysis.
Thus if a patient is randomized to surgery but dies while waiting
for surgery, that death is included as an event in the analysis.
Similarly, all operative events and postoperative deaths must be
included.
It is inevitable that some patients do not complete the trial for
whatever reason. For the trial to have internal validity, the number of protocol deviations must be minimized. In surgical trials
there is a greater risk that patients who are randomized to one
treatment may not receive it or may receive the alternative treatment. Steps to minimize this alteration must be instituted. For
example, patients often have a preference for one therapy or
another. Although an investigator is always anxious to accrue as
many patients as possible, it is important that the patient is willing
to accept either treatment. If not, if such patients are randomized
to the less preferred treatment, they may withdraw from the trial
and seek the preferred treatment outside the trial situation. Second, in a trial comparing two surgical therapies, patient consent is
obtained preoperatively but randomization is performed intraoperatively only after it has been determined that it is feasible to
perform both procedures. Again this method minimizes the risk of
crossovers. If a crossover does occur, the patient should be analyzed in the group to which he or she was randomized. Finally,
even if the need to assess whether a procedure is feasible is not an
issue, randomization should be delayed until immediately preoperatively to decrease the chance of adverse events occurring between the time of randomization and surgery or subjects changing
their minds about participating in the trial, especially if one
anticipates a delay in surgery.
Potential crossovers may not necessarily be considered crossovers depending on the trial design. One possible trial question
might be “Is surgery better than medical therapy?” If patients in
the medical group require surgery, rather than considering them
as crossovers they might be considered treatment failures, with
surgery being an outcome measure. In a second scenario the
question might be “Is early surgery better than medical therapy
followed by surgery if medical therapy fails?” rather than “Is
surgery better than medical therapy?” The former is a relevant
clinical question and mimics usual care in many instances. If this
were the question, obviously surgery is part of the “treatment
World J. Surg. Vol. 23, No. 12, December 1999
package” for the second group; and if surgery is performed in any
of these patients, they would not be considered crossovers. The
results would be analyzed based on the primary outcome measure
that had been chosen a priori.
Ensuring Adequate Accrual of Patients
Patient recruitment is often difficult no matter what RCT is
undertaken, but for surgical trials it may be even more difficult.
Even for diseases with a high incidence or prevalence, it may be
difficult to accrue patients because only a subgroup of the population are eligible. Surgical therapy may be indicated in only some
patients with the disease. Some patients may have co-morbidities
that preclude surgery. There may be different manifestations or
stages of the disease, and the entry criteria may restrict inclusion
to only some of the patients with the disease. For instance,
although cancer of the colon and rectum is the third commonest
cancer in North America, few patients would be eligible for a trial
comparing abdominoperineal resection to local excision. Similarly, although peripheral vascular disease is relatively frequent,
relatively few patients have isolated stenosis of the iliac artery
amenable to stenting or bypass procedures.
Patients may also have strong preferences for one therapy or
another and refuse to participate in a trial. For instance in the
NSABP trial comparing mastectomy to lumpectomy, accrual was
hampered because patients often had strong preferences for one
of the procedures [12]. Finally, there may be few surgeons experienced in performing a procedure (e.g., laparoscopic colectomy).
Because patient accrual is the most common reason for failure
of a trial, proper planning before the trial starts is required to
minimize the likelihood of failure on this basis. Successful completion is more likely if a multicenter rather than a single center
trial is undertaken. Adequate funding is mandatory to allow hiring
of research assistants who can ensure that all eligible patients are
approached to participate. During the planning stages patient
accrual should be realistically estimated taking into consideration
the number of patients who have the condition seen at each
institution each year, the proportion who would be eligible, and
the proportion who would consent to participate. Researchers
tend to overestimate the number of patients available for study
and who will agree to participate. Patient participation is variable
but often ranges between 10% and 50% of those eligible depending on the treatments, the intensity and length of the follow-up,
and the invasiveness of the follow-up investigations. The number
of participating centers and the duration of patient accrual may be
determined based on these estimates.
With medical trials, because of regulatory agency restrictions it
is unusual that the new therapy is available outside the trial, which
may be an incentive for patients to participate in the trial. Because
there are usually no restrictions on surgical therapies, there may
be less incentive for patients to participate in a surgical trial. The
investigators therefore might consider not performing the new
surgical procedure outside the trial. This decision must be made
considering the resources available and the ethics of restricting
the procedure to the trial.
Patient Issues
As discussed previously, patient preferences may limit the feasibility of a surgical trial. In a medical trial, patients may be ran-
McLeod: Issues in Surgical RCTs
domized to one arm or the other with the possibility that at the
conclusion of the trial they can receive the more efficacious treatment if the disease is not progressive and the treatment is reversible. Surgical procedures, however, are almost always permanent,
so patients will not have an opportunity to benefit from the results
of the trial. For example, although anterior resection is generally
favored over abdominoperineal resection in patients with rectal
cancers if it is technically possible, concerns exist that the local
recurrence rate may be higher after anterior resection. Despite
this concern, an RCT has not been performed, probably because
of the reluctance of patients to be randomized to receive a permanent colostomy with no hope of having it reversed if anterior
resection were found to be equivalent or superior to abdominoperineal resection. Furthermore, as in this situation, if the treatments are unequal, it may be even more difficult to accrue patients
to participate than in situations where the treatment alternatives
are perceived to differ little in their magnitude. Generally, when
there is a medical option or a less radical or invasive surgical
procedure being compared to another surgical procedure, patients tend to have definite preferences regarding therapy [12].
Additionally, accrual may be hindered because there is a lot of
emotion associated with surgery itself, and patients often are
preoccupied with these other issues. Moreover, they may be reluctant to leave the decision as to what therapy they will receive to
chance alone. Other issues, such as patients feeling like a “guinea
pig,” are common to both medical and surgical trials but again
may be of greater significance in surgical trials.
Obviously, patients must be allowed the freedom to decide
whether they want to participate in a trial. Investigators must
recognize some of these issues so they can realistically estimate
how many patients will consent to participate. Solicitation of
patient opinions may be helpful when planning trials and thus
increase accrual [13].
Surgeon Issues
Unlike the release of new medicines, surgery is not controlled by
a regulatory body such as the Food and Drug Administration
(FDA) or Health Protection Branch (HPB). Whereas all medications must undergo extensive investigation prior to their release,
new surgical procedures can often be performed with little or no
constraint even from the hospital or local ethics committee. Indeed, there may be a number of disincentives to surgeons wishing
to undertake a trial to assess a new procedure. First, if a trial is
instituted, approval must be obtained from the local ethics committee, a much more detailed consent form would probably be
required, the surgeon would have to discuss randomization with
the patient, and follow-up for the patient would probably be
intensive. On the other hand, if the surgeon were simply to
perform the operation, although a consent form would be required it would almost certainly be less inclusive. Economic disincentives may make participation in trials unattractive. Although
a physician would be compensated equally whether a patient was
or was not entered in a trial, this would not be the case for a
surgeon if the patient were randomized to a medical treatment
arm. Surgeons might also fear loss of their referral base if they did
not embrace a new procedure even though there was concern
about its effectiveness. Such was the case with the introduction of
laparoscopic cholecystectomy when general surgeons thought
their practices were threatened if they did not quickly adopt this
1213
new technology. Finally, funding of surgical trials is often a difficult issue. Pharmaceutical companies, because of pressure from
regulatory agencies to perform trials prior to release of medication, are a major source of funding for medical trials. Generally,
industry funding is less accessible for surgical trials.
Finally, it must be admitted that surgeons seem to place greater
reliance on the results of case series. Whether this is because of
the historical role of these studies, skepticism about the need for
RCTs, or the perceived difficulty of performing surgical clinical
trials, it is difficult to know with certainty.
Again, many of these issues can be obviated by performing
multicenter RCTs and including only surgeons who are committed to resolving the question and performing the trial. Accrual of
patients by surgeons who are not committed to a trial because of
ethical or financial concerns is almost always poor. To participate
in a trial successfully, investigators must have equipoise about the
treatments. To alleviate some of the concerns regarding a surgeon’s practice, it is worth noting that surgeons who are involved
in trials are often seen as experts in the field and patients preferentially want to be referred to them. Thus rather than detract
from their practice, it may in fact enhance it. Second, as resources
become more limited, surgeons will feel more pressure to evaluate
procedures before they are accepted. Finally, as evidenced by this
series of articles, there is a greater awareness of the need for
expertise in this area and greater academic prestige for performing good clinical research.
Ethical Issues
It is often said that it is not ethical to randomize a patient in a trial
where one treatment option is surgical. In fact, what is more
accurate is that it may not be feasible as discussed previously in
the section on “Patient Issues.” So long as there is community
equipoise, the patient is well informed, and informed consent is
obtained, a trial comparing a surgical to a medical alternative is
ethical. Successful completion of many important trials comparing
medical and surgical therapies is witness to this [5, 6, 10].
For a trial to be ethical, it is a given that the subject must not be
exposed to undue harm or risk. In addition, the trial must be
clinically relevant and significant. If not, it is not ethical to solicit
patients’ participation even if there is little risk involved. Surgeons, like other investigators, must be certain that they do not
have any conflict of interest when performing the trial. Trials
assessing new technologies are often industry driven, and surgeon
investigators may be financially rewarded for each patient accrued
into the study. In such situations, the surgeon may not be objective
when discussing the trial with patients and obtaining their consent.
Therefore it may be preferable to have a third party involved in
this aspect of the trial. Some universities have developed guidelines for obtaining consent in these circumstances. Finally, even
before embarking on such a trial, surgeon investigators should
believe that the trial has merit, is answering a significant clinical
question, and there is uncertainty with respect to the effectiveness
of the treatment options. Financial gain should not be the reason
for participating in a trial.
The issue of third party consent is more likely to be encountered in surgical than medical trials. In trials performed in emergency situations, on trauma victims, or patients in the intensive
care unit, the subject may not be able to sign his or her own
informed consent, and it may be necessary to rely on third party
1214
consent. This issue is discussed in a later article, but suffice it to
say that local ethics committees vary in their opinions regarding
third party consent.
Conclusions
There is agreement that the RCT is the best trial design for
determining treatment effectiveness. Although it has been widely
accepted for evaluating medical therapies, it has been used much
less extensively for evaluating surgical procedures. Many cite the
reason for this as methodologic issues related to surgical trials. As
discussed in this article, although there are challenges when performing surgical RCTs, in most cases they do not preclude their
performance. It is the author’s view that the reason for the
propensity of medical RCTs is the requirement by federal regulating bodies to provide clinical trial evidence before medications
are released. With this comes financial assistance from pharmaceutical companies to perform trials to evaluate medical therapies. Until now, surgical procedures have been unrestricted, with
little pressure and financial assistance available to evaluate them
in clinical trials. In the future, however, there will be increasing
pressure, not from regulating agencies such as the FDA or HPB
but by third party payors or governments who will refuse to fund
procedures or technologies without evidence that they are both
effective and cost-effective. Thus surgeons will be forced to take
up the challenge of evaluating surgical procedures more rigorously.
Résumé
Au fur et à mesure que la technologie se développe et que les
ressources de santé sont plus limitées, on assiste à une pression
croissante de la part des chirurgiens à évaluer leurs procédés
chirurgicaux et leurs technologies pour s’assurer qu’ils sont
efficaces. Ainsi, l’évidence fournie par des cas cliniques isolés n’est
plus acceptable et toute thérapeutique doit être évaluée par des
études cliniques contrôlées. Le point fort d’une étude contrôlée
par randomisation est qu’en vertu de sa conception rigoureuse, le
risque d’erreur (biais) que le résultat observé soit du au hasard ou
pas, est minimisé et ainsi, le risque de conclure à tort sur
l’efficacité d’un traitement quelconque est minimisé. L’étude
contrôlée par randomisation a été largement acceptée pour
évaluer l’efficacité d’un traitement médical mais l’est beaucoup
moins pour un procédé chirurgical. Beaucoup citent des
problèmes de méthodologie inhérents à la chirurgie pour
expliquer ceci. Ces problèmes sont la standardisation des
procédés chirurgicaux, comment choisir le moment propice des
essais en chirurgie, les difficultés pour «aveugler» les patients et
ceux qui évaluent les résultats, certaines questions d’éthique ainsi
que des problèmes d’acceptation du point de vue du patient et du
chirurgien. Ces notions sont discutées dans cet article. On évoque
également les moyens pour minimiser leurs effets négatifs.
Resumen
En la medida que se expande el uso de la tecnologı́a y los recursos
para la salud se hacen más limitados, se produce una creciente
presión sobre los cirujanos en el sentido de evaluar tanto los
World J. Surg. Vol. 23, No. 12, December 1999
procedimientos quirúrgicos como la tecnologı́a para determinar
su efectividad. El poder del estudio controlado y randomizado
reside en que por razón de su riguroso diseño, se minimiza el
riesgo de un error aleatorio o sistemático (sesgo) y con ello se
reduce al mı́nimo el riesgo de llegar a una conclusión errónea
sobre la eficacia de un tratamiento. El estudio controlado y
aleatorizado ha sido ampliamente aceptado para la valorización
de la eficacia de los tratamientos médicos, aunque no ası́ para la
de los tratamientos quirúrgicos. Como explicación de tal
fenómeno, muchos citan argumentos de tipo metodológico
relacionados con asuntos de naturaleza quirúrgica, tales como lo
pertinente a la estandarización del procedimiento operatorio, a la
cronologı́a de los ensayos quirúrgicos, a las dificultades para hacer
ciego tanto al sujeto como al investigador, a la ética de los ensayos
quirúrgicos y a la aceptación tanto por parte del paciente como
del cirujano. Tales aspectos son los que se discuten en el presente
artı́culo, junto con las posibles estrategias para minimizar su
efecto.
References
1. Eriksson, S., Granstrom, I.: Randomized controlled trials of appendectomy versus antibiotic therapy for acute appendicitis. Br. J. Surg.
82:166, 1995
2. Barie, P.S., Vogel, S.B., Dellinger, E.P., Rotstein, O.D., Solomkin,
J.S., Yang, J.Y., Baumgartner, T.F.: A randomized, double blind
clinical trial comparing cefepime plus metronidazole with imipenem
cilastatin in the treatment of complicated intraabdominal infections:
cefepime intra-abdominal infection study group. Arch. Surg. 132:1294,
1997
3. Lichtiger, S., Present, D.H., Kornbluth, A., Gelernt, I., Bauer, J.,
Galler, G., Michelassi, F., Hanauer, S.: Cyclosporine in severe ulcerative colitis refractory to steroid therapy. N. Engl. J. Med. 330:1841,
1994
4. Chalmers, T.C.: Randomization of the first patient. Med. Clin. North
Am. 59:1035, 1975
5. Friedman, L.M., Furberg, C.L., DeMeto, D.L.: Fundamentals of Clinical Trials (2nd ed.). Boston, PSG Publishing, 1985
6. North American Symptomatic Carotid Endarterectomy Trial Collaborators: Beneficial effect of carotid endarterectomy in symptomatic
patients with high grade carotid stenosis. N. Engl. J. Med. 325:445,
1991
7. Schmitz, R., Rohde, V., Treckmann, J., Shah, S.: Randomized controlled trial of conventional cholecystectomy versus minicholecystectomy. Br. J. Surg. 84:1683, 1997
8. Majeed, A.W., Troy, G., Nichol, J.P., Smythe, A., Reed, M.W.R.,
Stoddard, C.J., Peacock, J., Johnson, A.G.: Randomized, prospective,
single blind comparison of laparoscopic versus small incision cholecystectomy. Lancet 347:989, 1996
9. Dimond, E.G., Kittle, C.F., Crockett, J.E.: Evaluation of internal
mammary artery ligation and sham procedure in angina pectoris.
Circulation 18:712, 1958
10. Spechler, S.J.: Comparison of medical and surgical therapy for complicated gastroesophageal reflux disease in veterans. N. Engl. J. Med.
326:786, 1992
11. Peto, R., Pike, C., Armitage, P., Breslow, N.E., Cox, D.R., Howard,
S.V., Mantel, N., McPherson, K., Peto, J., Smith, J.G.: Design and
analysis of randomized controlled trials requiring prolonged observation of each patient. I. Introduction and design. Br. J. Cancer 34:585,
1976
12. Taylor, K., Margolese, R.G., Soskolne, C.L.: Physicians’ reasons for
not entering eligible patients in a randomized clinical trial of adjuvant
surgery for breast cancer. N. Engl. J. Med. 310:1363, 1984
13. Kennedy, E.D., Blair, J.E., Ready, R., Wolff, B.G., Steinhart, A.H.,
Carryer, P.W., McLeod, R.S.: Subjects’ perceptions of their participation in a clinical trial. Can. J. Gastroenterol. 12:287, 1998