Download 5 Design of Medical Experiments

5
Design of Medical Experiments
There are two broad types of study in medical research: observational and experimental. In this
chapter we will look at the design of experimental studies. These experiments may be performed
in the laboratory, on animals or on humans; trials of treatments on human subjects are called
clinical trials.
5.1
Medical Ethics
Ethics is the study of moral principles, thus medical ethics is the study of moral principles which
apply in the field of medicine. The medical statistician is part of the medical process and thus
needs to have an understanding of the ethical framework within which medicine is practised. This
section will outline the basic moral principles which are taken to apply in conducting scientific
(medical) research on humans. Thus when we design a study of humans we should expect that
these experiments meet all the requirements of these moral principles. We take morality to refer to
those social conventions about right and wrong human conduct that are so widely accepted by any
group (usually country) that they form an enduring consensus. It is not necessary that everyone
believes in the absolute rightness of these conventions, but it is expected that everyone abides by
them.
There are three main moral principles:
Autonomy The principle of autonomy requires that every person can only agree to take part
in a medical study provided they have had all the ramifications of taking part explained to
them and that they understand these ramifications. This is known as informed consent. This
principle also requires that they will suffer no adverse consequences if they refuse. This is
known as no duress. Also, once having agreed to participate, a person is free to withdraw
at any time and suffers no adverse consequences other than those which directly result from
the decision to withdraw.
Non-maleficence The principle of non-maleficence requires that harm is not deliberately done,
whether by acts of commission or omission. Thus in a medical study we are not allowed to
harm a patient just because we believe that we would learn something from such an action.
Beneficence The principle of beneficence requires that we contribute to the welfare of people.
Firstly, if we are offering a patient an experimental treatment we must have good reason to
believe that it is at least as good as the conventional treatment. Also we must design our
studies in such a way that that we gain the maximum information from them, thus providing
the maximum benefit to our subjects and the rest of society.
5.1.1
Medical ethics committees
While we have an absolute duty to abide by these principles, adherence to them is seen to be
so important that all research proposals are scrutinised by an ethics committee. The proposed
research is described in a document, called the research protocol, in sufficient detail to demonstrate
how it meets the principles; in particular it explains how the research will be conducted and
analysed. The proposed analysis is contained in an analytical protocol. Part of the construction
of a research protocol and its scrutiny by an ethics committee requires considerable technical
expertise, especially in the areas of epidemiological method and statistics. Thus both consultant
epidemiologists and statisticians should play a key part in these processes.
In addition, the research protocol forms a handbook to guide those conducting the study. The
protocol usually has the following sections:
26
1. Rationale: the background to the study.
2. Aim of study: exactly what are the experimenters trying to find out.
3. Design of the study: the epidemiological method used and the methods used to avoid bias.
Sample size calculations.
4. Study population and selection of patients, including exclusion criteria.
5. The intervention, specified in sufficient detail to allow another clinician to repeat it.
6. Methods of assessment of the success or otherwise of the intervention. Outcomes should be
divided into the primary outcome measure, which is the main way the intervention will be
judged, and secondary outcome measures which add further data on the performance of the
intervention.
7. Documentation: Patient information sheets and consent forms, data sheets etc.
8. The procedure to be adopted, including all decisions to be taken.
9. Withdraws: while patients may elect to leave the study at any time the experimenters must
know the circumstances when a subjects best interest are served by withdrawing them from
the experiment.
10. Adverse events: their description and consequent decision making.
11. The process of consenting patients.
12. Ethics: The name of the ethics committee(s) consulted.
13. Analytical protocol.
14. Confidentiality: how the data will be managed to ensure patient confidentiality.
15. A statement of conflicts of interest and declaration that the protocol will be followed.
In large multi-centred studies a data monitoring committee is set up to oversee the data, and to
ensure that no unexpected problems are arising with any of the treatments. The experimenters
cannot do this as partial knowledge of the results can subconsciously influence the decisions they
take. The role of the statistician is pivotal here to determine if the interim results of a study are
fully informative, especially where repeat statistical testing takes place.
Once the research is completed the results should be placed in the public domain; the experiment
is written up as a paper and submitted to a medical journal. The paper is sent by the journal
to several independent reviewers who check on the validity of the experimental process and the
legitimacy of the conclusions. This process is called peer review. Most journals have statistical
advisors to help the editor deal with some of the more complex or less common statistical issues
that arise. The publication of the results of a study should not be taken as confirming that the
results represent the ‘true state of nature’. Rather the publication confirms that the study has
been conducted and written up to an appropriate standard.
5.1.2
Statutory procedures in new drug testing in the UK
The testing of new drugs is very tightly regulated; a number of criteria need to be met before a drug
is licensed for routine use. There are four stages or phases of drug testing on humans. This testing
on humans follows both in vitro (laboratory testing, often on tissue) and animal experimentation
(which provokes further ethical deliberation). The four phases, which reflect the different stages
of human experimentation, are:
27
Phase 1 trials Testing of drug safety and basic dose response (pharmaco-kinetics). Usually done
on a small number of healthy volunteers using sub-therapeutic doses.
Phase 2 trials The first experiments on patients, that is people with the condition that the drug
is intended to treat. These studies try to identify the optimum therapeutic dose, balancing
benefit against side effects. Careful monitoring of the subjects is required, often involving
a wide range of measurements. The number of subjects involved is small, but usually more
than phase 1 studies.
Phase 3 trials These are the large scale comparative clinical investigations which take place
within ordinary medical practice. This is the most formal of the phases and can involve
thousands of patients. Once phase 3 data are available the drug can be reviewed by the
regulatory bodies who can issue a product licence so that the drug can be marketed. This
phase is often referred to as clinical trials.
Phase 4 trials This is the post-marketing surveillance and is concerned with the long term safety
and efficacy of the drug. For consistency of terminology these studies are called trials but
strictly they are surveys rather than experiments.
5.2
Randomised controlled trials
Randomised controlled trials, or RCTs, are the gold standard for the assessment of medical interventions. While they have their critics and have a number of known disbenefits they give results
which are far easier to draw safe inferences from than any other type of experimental design. All
of the key design features of RCTs are intended to remove bias; that is, systematic differences
between the two groups to be compared.
Blind recruitment The eligibility of a patient is assessed without knowing to which of the treatments under study he/she will be allocated. The patient is then admitted to the trial only if
there is no preference between treatments A and B for this particular patient.
Random allocation to treatment Where the comparison groups are not selected at random,
one group may be intrinsically favoured to do well, irrespective of the treatment they receive.
If we randomly allocate patients we are seeking to avoid any overall bias in favour of one
treatment (we are not trying to make the two groups equal, just avoid bias). Groups are
comparable in that differences between groups are no greater than would be observed through
chance.
Patients blinded to treatment For reasons that are not always clear patients can tend to improve more on one type of treatment compared to another, even when it is known that the
treatments are identical. This applies to both subjective symptoms such as degree of pain,
and even apparently purely physiological measures such as blood pressure. It is a matter of
vigorous ethical debate as to the steps which can be taken to preserve this blindness.
Experimenters blinded to treatment Despite strong protestations to the contrary by researchers,
knowledge of the treatment that the patient is receiving can influence decisions taken. Similar circumstances apply during the assessment of subjects, such as a clinician’s subjective
opinion on a patient’s progress.
With drug treatments it is usual for only the pharmacist to know which treatment each
patient is taking.
Analysis blind to treatment If the task of managing the data is done by a different person to
the one who will analyse it there is no reason why the analysis and statistical interpretation
cannot be done blind to the treatments. Care needs to be taken when attempting this as
often a treatment group can be identified by the characteristics of the data e.g. the pattern
of adverse events.
28
Table 5: Results of the field trial of Salk polio vaccine for two different designs
Number
Treatment
Group
Randomised Control Trial
Vaccinated
200745
Refused Vaccination
338778
Control
201229
Non-randomised
Vaccinated
221998
Refused Vaccination
123605
Control
725173
5.2.1
Cases of
Polio
Rate per
100,000
33
121
115
16
36
57
38
43
330
17
35
46
Intention-to-treat analysis
The procedures outlined above ensure that the groups who are allocated to receive the various
treatments or placebo start off having no systematic bias. Not all patients will continue with the
specified intervention or placebo until the endpoint of the study. There must be the option to
adjust the patient’s treatment should clinical circumstances necessitate it, or patients may also
decline to continue following the treatment they were randomised to. There are two strategies that
can be used; the choice depends on the objectives of the study.
Studies of therapeutic efficiency If we are interested in therapeutic efficiency (that is, does the
treatment do what it is intended to do) then we only analyse the subjects who stayed on the
treatment they were randomised to and we report the others as either protocol violators or
patient withdraws. Thus the results will tell us the comparative advantage of one treatment
over another for patients who complete the treatment. This is known as on treatment analysis.
Studies of treatment policy In other cases, the ‘treatment’ may not be the medical intervention
itself, but a policy of offering it to patients and treating those who accept; for example,
vaccinations, health checkups. In this case we analyse the data according to the way we
intended to treat subjects, rather than the way in which they actually were treated. This is
known as analysis by intention to treat, and answers questions of policy effectiveness.
5.2.2
Cross-over designs
Sometimes it is possible to use patients as their own controls so that each subject receives all
treatments over different periods of time. Such designs are called cross-over designs and are
advantageous since they remove variability between subjects. They also require fewer subjects
than a two group trial. Treatment order should be assigned at random, and sufficient time allowed
between treatments to allow any effect of the first treatment to dissipate.
Cross-over designs are only suitable in clinical trials where the treatment will not affect the course
of the disease and where the patient’s condition will not change appreciably over the course of the
trial. They can also not be used to demonstrate long-term effects, or for treatments that have a
long-term impact after discontinuation.
29
5.2.3
Deficiencies in the RCT
The RCT can be criticised because of the highly stylised structure, and its inherent limited ability
to test many scenarios or many different outcome measures at once. While these are legitimate
criticisms there is no obvious alternative investigative model known.
There are three cornerstones of the RCT design namely random allocation, patients blinded to
treatment and assessors blinded to treatment. These features protect us from finding differences
due to bias. They do not protect us from finding no difference due to bias. Thus any result from
an RCT which finds ‘no difference’ could be due to bias unless very careful checks were made.
5.3
Non-randomised Designs
Randomised controlled trials are not always possible or ethical. A cohort study can be used as a
non-randomised design for experiments. A cohort is a well defined population, usually followed-up
for a period of time.
5.3.1
Pre-test/Post-test Studies
A pre-test/post-test study is a single cohort study in which a group of individuals are measured,
then subjected to a treatment or intervention, and then measured again. The purpose of the study
is to observe the size of the treatment effect. Deficiencies in before and after comparisons arise
because serial changes in a single group of subjects are studied. Any changes seen over due to
changes other than the treatment over the time period.
Period effects There may be reason to expect systematic differences between the values recorded
at two times that have nothing to do with the effect being studied. If the two times of
assessment are separated by six months, and all patients enter the study in summer, the
treatment effect will be confounded with a seasonal effect. If the study period is several
years there may be an ageing effect.
Placebo effects The fact that the patient is aware of having a potentially beneficial treatment
administered or prescribed often has an effect. Placebo effects may also influence the observer,
especially when the measurement being made is largely subjective.
The Hawthorne Effect This is similar to the placebo effect except that the cause of the improvement is just taking part in a study.
Regression towards the mean As in Section 2.2.4, this occurs when subjects are selected on
the basis of having high levels of a relevant test measurement.
5.3.2
Comparative cohort
In a comparative cohort design, we compare two groups as before, one group experiencing effect
A, the other effect B. Our study is then a controlled trial.
We can sometimes use historical controls. For example, when there is a national change in health
care policy or available treatment, the new incidence of the condition of interest can be compared
with the old incidence before the change. Again, this may introduce bias due to other differences
before and after the change.
30