Download Comparison of Hypertonic Saline to Epinephrine in

Proof of Concept Study
Comparison of Hypertonic Saline to Epinephrine in Bronchiolitis using CUSUM:
Use of a Novel Statistical Tool for Continuous Monitoring of New Therapy
Neeraj Gupta
Ashish Puliyel
Ayush Manchanda
Jacob Puliyel
Department of Pediatrics and Neonatology
St. Stephen’s Hospital,
Delhi – 110054
Address for Correspondence:
Neeraj Gupta
House No. 249 – E,
Near Police Station, Nawada Bazar,
Najafgarh, New Delhi – 110043
E-mail: [email protected]
Key words:
Bronchiolitis, CUSUM, stopping rule, control limit lines, Randomized Control Trials,
boot-strapping.
Author Contribution
NG, AP, AM and JP conceive the project. NG conducted the clinical trial. AP
helped NG and JP in the statistical analysis and development of software. NG, AM and
JP were responsible for the write up. JP will act as guarantor of the paper.
Funding – None
Competing interest – None
What is known?
Nebulised hypertonic saline has been found to be better than placebo in
bronchiolitis.
What this study adds
1. Nebulised hypertonic saline is at least as good as standard treatment with
nebulised Epinephrine.
2. A software for easy calculations of CUSUM has been developed that can help
monitor new therapies in real time.
Abstract
Background
Nebulised bronchodilators are used in treatment of severe cases of acute
bronchiolitis for the temporary relief it affords although it does not reduce hospital stay.
Nebulised hypertonic saline is a new mode of therapy for bronchiolitis but it has only
been studied compared to placebo, not against standard-of-care medication (used in our
hospital) namely nebulised Epinephrine.
Cumulative sum (CUSUM) is a statistical tool used for quality control in industry.
The use of this tool in medicine has also been reported recently. We have employed
CUSUM here to monitor in real time, if nebulised hypertonic saline is better or worse
than standard-drug Epinephrine in acute bronchiolitis.
Materials and Methods
35 children with first episode of bronchiolitis presenting in first half of a
bronchiolitis season received nebulised Epinephrine as the standard treatment. Heart rate
and treatment responses using the clinical scoring matrix of Uyan et al were monitored
over the next 24 hours. CUSUM analysis was then done using the data obtained from
Epinephrine group and control limit lines drawn in between which the outcome of new
drug was expected to fall if it was as effective as Epinephrine. For the remaining part of
the season, nebulised hypertonic saline was used instead of Epinephrine and outcome
monitored in real time by using CUSUM control lines.
Results
There was no statistical difference in clinical score, heart rate and duration of
hospital stay in between two groups. Nebulised hypertonic saline was as effective as
nebulised Epinephrine on CUSUM scoring in this limited study involving 64 patients.
Discussion
Previous studies compared nebulised hypertonic saline with placebo, either alone
or mixed with nebulised Salbutamol in both limbs. They found hypertonic saline useful in
the treatment of acute bronchiolitis. This is the first study comparing nebulised
hypertonic saline against nebulised Epinephrine. We found that nebulised hypertonic
saline was as good as standard-therapy in our hospital with Epinephrine.
We used CUSUM statistics to monitor the new drug in real time during the trial
and to provide us with a stopping signal. For this study we developed the software to
allow easy calculation of CUSUM. This novel technique can help to monitor outcomes
during a trial and advice on the need to stop a trial if the new drug performs worse than
the control drug. This therefore has potential for widespread use in clinical trials making
them safer for patients.
Proof of Concept Study
Comparison of Hypertonic Saline to Epinephrine in Bronchiolitis using CUSUM:
Use of a Novel Statistical Tool for Continuous Monitoring of New Therapy
Introduction
Acute bronchiolitis is a disease of the lower respiratory tract of infants, resulting
from inflammatory obstruction of small airways, characterized by tachypnoea, wheezing,
fine crackles and air trapping1. It is usually a self-limiting disease. However some
children are too breathless to feed and some require oxygen supplementation to maintain
blood oxygen levels. These children may require hospitalization. Treatment is mostly
supportive using fluids and oxygen therapy. Nebulised bronchodilators like Salbutamol,
Ipravent and Epinephrine have been used by some in treatment of bronchiolitis. A
Cochrane meta-analysis has not found these drugs to be useful2. In spite of the lack of
objective evidence for their usefulness, these drugs are used extensively. The outcome
measure used in this Cochrane meta-analysis was duration of hospital stay rather than
improvement in respiratory distress. Another Cochrane analysis looking for benefits in
terms of respiratory distress and oxygen requirement in immediate post treatment period
with bronchodilator therapy, found improvement3. These improvements are transient and
do not help in reduction of hospital stay. It is clear that it is for this temporary but
perceptible relief of symptoms that these drugs are used.
There is little consensus on the bronchodilator that is most useful. A study
published in Journal Pediatrics shown that both Epinephrine and Salbutamol reduces
respiratory rate but Epinephrine has better influence in reducing total pulmonary
resistance4. Epinephrine however has the disadvantage of increasing heart rate5. A
Cochrane review of the use of Epinephrine found evidence that it was more effective
when used in the outpatient setting but no evidence of benefit when used in inpatients
when compared against either placebo or Salbutamol3.
At our hospital we have been using nebulised Epinephrine in children with
bronchiolitis whose distress warrants treatment.
Nebulised hypertonic saline is a new modality of treatment that has recently been
used in bronchiolitis. It has previously been studied in the context of cystic fibrosis. Like
bronchiolitis, children with cystic fibrosis have viscid secretions and wheezing. The
hypertonic saline helps to reduce viscosity of secretions. A Cochrane meta-analysis of its
use in cystic fibrosis concluded that nebulised hypertonic saline improves mucociliary
clearance immediately after administration and that it may have a long term beneficial
effect6.
It is postulated that nebulised hypertonic saline may be beneficial in bronchiolitis
also. Cochrane database analysis, involving 4 trials with 254 infants, using hypertonic
saline mixed with bronchodilators, suggests nebulised 3% saline may significantly reduce
the length of hospital stay and improve the clinical severity score in infants with acute
viral bronchiolitis when compared with bronchodilator alone7. A systematic review
published in Indian Pediatrics looking at hypertonic saline against placebo found
significant reduction in duration of hospital stay in the hypertonic saline group in children
with bronchiolitis8. It seems that nebulised hypertonic saline with bronchodilators is
better than bronchodilators alone and that hypertonic saline is better than placebo, but
there is no study comparing hypertonic saline directly against bronchodilators. Ethical
standards now-a-days demand that when treatment exists that is better than doing nothing
for the patient, the alternative treatment must be standard therapy rather than placebo. In
our hospital standard therapy for children with severe disease is nebulised Epinephrine
and therefore it was considered appropriate to study nebulised hypertonic saline against
nebulised Epinephrine in hospitalized patients. Prior to start of study, we searched the
clinical trial register to look for trials registered with this protocol. Other than an ongoing
trial9 in United States using hypertonic saline against placebo, we found no trials
registered using hypertonic saline in bronchiolitis and there were none comparing
hypertonic saline to bronchodilators.
The best answer to this question of which of the two alternative intervention is
better can be studied in a double blind Randomized Control Trial (RCT) comparing the
new drug (nebulised hypertonic saline) against standard therapy (nebulised Epinephrine).
However, RCT’s have inherent problems, especially in the context of trials in children.
According to Mc Culloch and colleagues, RCT’s require large samples, long duration,
difficult blinding and are very expensive10 and it is difficult to recruit cases11. Parents
find the concept of equipoise between trial drugs and the need for blinded randomization
difficult to understand.
CUSUM (Cumulative Sum) is a statistical technique used in industry for quality
control. Limits called ‘control limits’ are set based on the probability of error or failure in
the standard process. Essentially, positive weights are given for successes and negative
weights are given for failures such that as the process continues, the cumulative score
stays close to the zero line. Using statistical calculations two control lines are drawn such
that, if there is a real change in the process (more failures or success than would be
expected by chance), it will cross the control line and the change can be detected early.
These control lines are drawn using a process of boot-strapping. The idea behind bootstrapping is that there is a random reordering of the same data as the sequence of failures
and success depends on chance. By repeating the boot strapping sequence in 10,000
iterations, the limits of the CUSUM that can occur by chance with this process can be
defined12. The highest CUSUM and the lowest CUSUM for each iteration is noted. The
upper limit of CUSUM is considered the mean upper CUSUM + 2 SD and the lower limit
is considered as the mean of the lowest values - 2SD of the mean.
CUSUM has been used in medical practice earlier. It has also been used for
plotting temperature charts for assessing antimicrobial treatment in neutropenic
patients13. Ravn LI and colleagues found that, the CUSUM analysis is a valid and
practical technique for the qualitative assessment of clinical competence as well as a tool
for monitoring continuous professional development14. Watkins RE and colleagues used
CUSUM based methods for the detection of outbreaks of Ross River Virus disease in
Western Australia15. We have used this novel statistical tool to see if hypertonic saline
was better or worse than the standard drug; Epinephrine. In this communication we report
the procedure, the software we developed and the results of the study. We have also
discussed how CUSUM can be used for other clinical trials even within the randomized
control trials.
Materials and Methods
This study was approved by the Hospital Ethics Committee (Minutes of
the Ethics committee meeting, St. Stephen’s Hospital, November 2008). It was registered
with Clinical Trials Registry India (CTRI/2008/091/000233). The study was conducted in
one bronchiolitis season (November 2008 to April 2009). Children, 2 months to 2 years,
presenting with first episode of acute bronchiolitis and respiratory distress to our hospital
emergency room were enrolled in the study after obtaining written informed consent from
parents. Children with history suggestive of chronic cardiopulmonary disease,
immunodeficiency, past history of respiratory disease requiring nebulisation and critical
illness at presentation were excluded. Children with a history of use of systemic or
nebulised bronchodilators or nebulised hypertonic saline in last 24 hours were excluded
from study. Clinical scoring system described by Uyan et al16 was used. The scoring
system is summarized in Table I. A score of 4 or more or oxygen saturation less than
94% on room air warranted admission. Enrolled children were treated as usual with
standard drug; nebulised Epinephrine. They received nebulised Epinephrine (1 mL
diluted in 2 mL normal saline) every 6 hourly for first 24 hours and were monitored using
the same scoring system. Heart rate during first 24 hours of therapy was tracked. Any
child having increase in clinical score of 2 or more (using admission score as baseline) or
if heart rate went above 200/minute and there was 10% increase over baseline, it was
considered as treatment failure. If the child improved or did not deteriorate using the
above criteria, the treatment was considered useful for the purpose of our study. In case
of failure, the drug was stopped and alternative measures were instituted which could be
escalated up to ventilation. Nebulised Epinephrine was given during the first 3 months of
study. CUSUM data was analyzed to look for failures in standard therapy and to draw
control limit lines. A series of 35 patients got the standard drug (nebulised Epinephrine)
treatment. Using boot-strapping technique and by random rearrangement of the sequence
we looked at the CUSUM scores in 10,000 iterations. This helped to look at the limits
that occur purely on account of chance changes in sequence of success. CUSUM scores
using the boot-strapping method provided the upper and lower control lines. The mean
upper CUSUM score + 2SD provided the upper limit and the mean lower CUSUM score
-2SD provided the lower limit. In this study the data was acquired before the CUSUM
software was fully developed. The pragmatic stopping rule we adopted while acquiring
the data was to temporarily stop the trail (pending full development of the software) if the
CUSUM with the new drug exceeded the overall rate of failure with the standard drug (if
the CUSUM of failures with the new drug crossed the zero line).
Subsequently nebulised hypertonic saline was introduced. All children enrolled
after that point in time, received nebulised hypertonic (3%) saline 3 mL every 6 hours.
Clinical score and heart rate were monitored. Success and failures were measured by the
same criteria as with Epinephrine. The CUSUM score for each child, given the nebulised
hypertonic saline, was plotted in real time. We expect the CUSUM plot to stay with in the
control lines if the hypertonic saline was as effective as Epinephrine. Apriory it was
decided that if failure rate crossed the CUSUM control line, the trial will be stopped
immediately. If however, successes were more and it crossed the upper control line, the
study would be continued till the end of bronchiolitis season and it would be clear that
new drug is superior to standard therapy. In the absence of suitable software, we
developed custom-built open-source software which allows inputting of any initial series
data (Epinephrine in the study). The software does the boot-strapping and calculates the
limits
for
the
control
line.
The
software
is
available
at
http://jacob.puliyel.com/calculator/. For looking difference between means, we used
statistical software CIA (Confidence Interval Analysis www.som.soton.ac.uk/cia).
Results
64 patients were enrolled in the study. 35 received nebulised Epinephrine and
subsequently 29 received nebulised hypertonic saline. Details of children in two groups
are given in Table II. There was no statistical difference in the age, sex distribution,
clinical score and heart rate at admission. The mean age (in months) was 7.157 (SD
6.582) in Epinephrine group and 5.276 (SD 3.827) in hypertonic saline group. The mean
clinical score at admission was 8.229 (SD 2.576) in Epinephrine group and 7.552 (SD
2.283) in hypertonic saline group. The mean heart rate at admission was 149/minute (SD
41) in Epinephrine group and 143/minute (SD 16) in hypertonic saline group. The heart
rate after 24 hours of treatment was also similar in two groups and there was no statistical
difference in the duration of stay.
With the initial trial with Epinephrine, there were 5 children with treatment failure
(2 with worsening of clinical score and 3 with tachycardia). The resultant weightage for
success and failure were + 0.285714 and – 1.714285 respectively. The highest CUSUM
during the trial using Epinephrine was + 5.428566 and the lowest CUSUM was –
0.571438. Figure 1 shows the CUSUM graph of success and failure with Epinephrine.
Boot-strapping was done 10,000 iterations. Iterations were repeated until further
iterations did not significantly alter the CUSUM maximum and minimum values (Figure
2 shows the standard CUSUM with 10 iterations). The maximum was + 2.25289 (SD
1.34236) and minimum was – 2.25866 (SD 1.33735). + 2 SD (standard deviation) of the
maximum and – 2 SD of the minimum values were used to draw the control lines as
shown in Figure 3.
Discussion
Using this technique, we found that nebulised hypertonic saline was at least as
good as nebulised Epinephrine in treatment of acute bronchiolitis. This suggests that
deterioration with in the hospital was no more frequent with hypertonic saline than with
Epinephrine. Previous studies using hypertonic saline have also found it more useful than
placebo (normal saline)7, 17 and hypertonic saline used with Salbutamol was better than
Salbutamol alone18. This is the first study directly comparing nebulised hypertonic saline
with nebulised Epinephrine. While using Epinephrine 3 of 35 patients developed
tachycardia and the drug has to be stopped. No child in hypertonic saline group
developed tachycardia.
We used the CUSUM statistics to monitor the new drug (nebulised hypertonic
saline) in real time with a stopping rule that allowed the study to be terminated in case the
new drug was less beneficial than standard therapy. This is a form of comparison against
historical control in as much as the study drug and the control drug were administered at
different time periods. The main criticism about using historical control is that other
changes (beside the new drug) may have been instituted after the data on historical
control was collected. In the case of our CUSUM study both drugs were used in sequence
in the same bronchiolitis season and there were no other differences in the treatment
modalities during two time periods. However it is still possible that bronchiolitis severity
may be different in the beginning of the season compared to later half. This is a drawback
of this methodology we used, but looking at the severity score at admission in the group
giving nebulised Epinephrine and in the group giving nebulised hypertonic saline, there
was no significant difference.
Weakness of the study
This is a one centre study done in one bronchiolitis season. The numbers of cases were
limited. A multicentric study would give more certain data. More studies and larger
numbers will be needed before definite conclusion can be made.
Strengths of the study
This novel technique of using CUSUM helps us to stop trial early if the trial drug
performs worse than standard therapy. It can be used for comparison against placebo or
against a standard therapy. The new tool can be used instead of RCT’s, or as a
preliminary trial run before RCT is planned, or it can be used within the RCT to allow for
continuous monitoring of the trial allowing early closure of RCT if it crosses control lines
(rather than after fixed periodic data analysis done by data monitoring committees). The
idea for a clinical CUSUM calculator was taken from industry (quality control
mechanism) to provide a simple method to monitor a drug trial. Paradoxically, industry
itself may find the simple software,we developed, handy for quality control.
Scope for further research
More research in large multi-centric double-blind trials is needed before nebulised
hypertonic saline can be recommended for general use in treatment for bronchiolitis. The
CUSUM technique we have used must also be evaluated in other trials before it can be
recommended.
Conclusion
We believe that the software that we developed for the study allows easy bootstrapping and drawing of control lines and thus has potential for use in many clinical
situations. We found that hypertonic saline performed as well as nebulised Epinephrine in
preventing deterioration of condition in bronchiolitis.
References
1. Goodman D. Bronchiolitis. In: Behrman RE et al eds. Nelson Textbook of
Pediatrics. Philadelphia: WB Saunders; 2004. p.1415-17.
2. Kellner JD, Ohlsson A, Gadomski AM, Wang EE. Bronchodilators for
bronchiolitis. Cochrane Database Syst Rev 2000;(2):CD001266
3. Hartling L, Wiebe N, Russell K, Patel H, Klassen TP. Epinephrine for
bronchiolitis. Cochrane Database Syst Rev. 2004;(1):CD003123
4. Sanchez I, De Koster J, Powell RE, Wolstein R, Chernick V. Effects of racemic
epinephrine and salbutamol on clinical score and pulmonary mechanics in infants
with bronchiolitis. J Pediatr. 1993;122:145-51
5. Wainwright C, Altamirano L, Cheney M, Cheney J, Barber S, Price D, Moloney
S, Kimberley A, Woolfield N, Cadzow S, Fiumara F, Wilson P, Mego S,
VandeVelde D, Sanders S, O'Rourke P, Francis P. A multicenter, randomized,
double-blind, controlled trial of nebulized epinephrine in infants with acute
bronchiolitis. N Engl J Med. 2003; 349:27-35
6. Wark PA, McDonald V. Cochrane Database Syst Rev. 2000;(2):CD001506
7. Zhang L, Mendoza-Sassi RA, Wainwright C, Klassen TP. Nebulized hypertonic
saline solution for acute bronchiolitis in infants. Cochrane Database Syst Rev.
2008 Oct 8;(4):CD006458
8. Joseph Mathew. What works in Bronchiolitis?. Eureca. Indian Pediatrics 2009;46
p 154-8
9. Nebulized Hypertonic Saline for Bronchiolitis. NCT00619918
10. McCulloch P, Taylor I, Sasako M, Lovett B, Griffin D. Randomised trials in
surgery: problems and possible solutions. BMJ 2002;324:1448-51
11. Yeung V. Clinical trials in children. In Pediatric drug handling ISBN
9780853696865 accessed on 12/10/2009 available at
http://www.pharmpress.com/shop/samples/paeditaric_sample_chapter.pdf
12. Wayne A. Taylor. Change-Point Analysis: A Powerful new tool for detecting
changes.
Web:http://www.variation.com/cpa/tech/changepoint.html
accessed
11/10/09
13. Kinsey SE, Giles FJ, Holton J. Cusum plotting of temperature charts for assessing
antimicrobial treatment in neutropenic patients. BMJ 1989;299:775-6
14. Ravn LI, Sprehn M, Pedersen CB. The Cusum score. A tool for evaluation of
clinical competence. Uqeskr Laeqer 2001 Jun 25;163(26):3644-8
15. Watkins RE, Eagleson S, Veenendaal B, Wright G, Plant AJ. Applying cusumbased methods for the detection of outbreaks of Ross River virus disease in
Western Australia. BMC Med Inform Decis Mak. 2008 ;13;8:37
16. Uyan AP, Ozyurek H, Keskin M, Afsar Y, Yilmaz E. Comparison Of Two
Different Bronchodilators In The Treatment Of Acute Bronchiolitis. The Internet
J of Peditr & Neonat 2003;3(1)
17. BA Kuzik, SA Al Qadhi, S Kent, MP Flavin, S Hotte, S Gander: Nebulized
hypertonic saline in the treatment of viral bronchiolitis in infants. J Pediatr. 2007
Sep;151(3):266-70, 270.e1. Epub 2007 Jun 29
18. Luo Z, Liu E, Luo J, Li S, Zeng F, Yang X, Fu Z. Nebulized Hypertonic
Saline/Salbutamol Solution Treatment in Hospitalized Children with Mild to
Moderate Bronchiolitis. Pediatr Int. 2009 Aug 7
Table I - Clinical Scoring System by Uyan et al 16
Score
0
1
2
3
Breath Rate
< 30
30 - 45
46 – 60
> 60
No
Only
Intercostals,
Abdominal
intercostals
subcostal &
respiration
supraclavicular
accompanying
Moderate to
Severe &
during
Severe &
continuously
inspiration
intermittently
(/min.)
Retractions
Nasal flaring
Wheezing
No
No
Mild & rarely
Heard only
Heard in both
Heard in both
with
phases of
phases of
Stethoscope
respiration with
respiration
Stethoscope
without
Stethoscope
General Status
Normal
Moderately
Very uneasy,
uneasy &
crying
occasionally
continuously
crying
Score 4-8 = moderately ill; 9 or more = severely ill
Lethargic
Table II – Population Characteristics
Patient
Nebulised
Nebulised
95% CI for
characteristic
adrenaline
hypertonic saline
difference between
means
No. of patients
35
29
Male: Female
25:10
23:6
Age (in months)
7.157 (6.582)
5.276 (3.827)
-0.886 to + 4.646
Clinical score at
8.229 (2.576)
7.552 (2.283)
-0.549 to + 1.909
149 (41)
143 (16)
-10.168 to + 22.168
3.516 (2.816)
3.552 (2.686)
-1.416 to + 1.356
136 (44)
137 (12)
-17.849 to + 15.849
96.029 (111.41)
82.914 (65.940)
-33.888 to + 60.128
admission
Heart Rate at
admission (/min.)
Clinical score at 24
hours
Heart rate at 24
hours (/min.)
Duration of stay
(hrs)
Figure 1
CUSUM graph of success and failures with nebulised Epinephrine
8
6
4
CUSUM score
2
0
Series1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
-2
-4
-6
-8
Series of Patients
Figure 2
10 illustrative boot-strapping iterations of CUSUM
8
6
4
CUSUM score
2
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
-2
-4
-6
-8
Series of Patients
Figure 3: Real time CUSUM plot with nebulised Hypertonic Saline with CUSUM control
lines