Download ACE 130 reviewer#1. defense RCT merge crit.

CARG 130 reviewer#1. defense RCT merge criteria.summary.docx
Page 1 of 7
Reviewer #1 June 4, 2011 and another expert opinion summary
Background to the development of criteria to merge RCTS
Abstract of analysis of comparison 1 to 5
There was clinical heterogeneity between the merged RCTs in comparison 1 to 4. Experts have stated
that RCTs like these should not be merged for a meta-analysis because of this type of clinical
heterogeneity. Some experts believe that meta-analysis should only include trials that have similar
patient groups, using a similar intervention, similar classes of drugs and measure similar endpoints
(Myles 2011, Avidan 2013).
Broad vs narrow review question
Other experts state that the use of a set of identical studies to synthesize data is the least common goal
of a meta-analysis. More common goals to perform a meta-analysis are to “broaden the base of studies
in some way, expand the question, and study the pattern of answers…The goal of some syntheses will
be to report the summary effect, but the goal of other syntheses will be to assess the dispersion as well
as the mean effect, and the goal of others will be to focus on the dispersion exclusively. “A good metaanalysis will anticipate this diversity and will interpret the findings with attention to the dispersion of
results across studies.” (Borenstein, Hedges et al. 2009).
More narrow inclusion criteria to study processed EEGs would be appropriate for FDA approval of a new
medical device but a broader inclusion criterion would be appropriate for most meta-analyses compared
to the primary studies it includes.
Meta-analysis findings may not only be quantitative but also may be qualitative and reveal the biases,
strengths, and weaknesses of existing studies. The results of a meta-analysis can be used to form
treatment recommendations or to provide guidance in the design of future clinical trials (Russo 2007).
We anticipated this diversity or clinical heterogeneity by adjusting our statistical analysis as described in
the Data synthesis section of CARG 130.
If we want to answer the question, do processed EEGs decrease the frequency of awareness across a wide
range of clinical circumstances, then allowing significant clinical heterogeneity is acceptable “as long as
we address that issue in our interpretation of the results of the meta-analysis..” (Borenstein, Hedges et
al. 2009) For example, we want to know if processed EEGs can be an effective intervention in both high
and not high risk awareness patients, in minor, moderate and major risk surgical procedures, in a wide
age range of patients and using abroad range of anesthetic techniques
Classification: Intravenous and volatile agent based techniques
Intravenous techniques
The argument that follows is based on two definitions of general anesthesia. The first definition is
traditional intravenous anesthesia, which includes total intravenous anesthesia (TIVA) techniques prior to
the introduction of propofol remifentanil based TIVA. This definition was hypnosis, analgesia and
CARG 130 reviewer#1. defense RCT merge criteria.summary.docx
Page 2 of 7
relaxation. The relaxation implies that neuromuscular blocking drugs (NMBs) are required to ablate
movement or induce abdominal muscle relaxation.
Volatile agent techniques
The other definition that describes volatile agent based techniques is unconsciousness, analgesia and
optional neuromuscular blocking agents (NMBs) use for the same two indications as with IV anesthesia.
The optional nature of NMB use in volatile agent based techniques is because volatile agents are a total
anesthetic compared to IV agent components. That is, volatile agents have hypnotic sedative effect in low
doses and induce unconsciousness in higher doses. They have analgesic action. They induce abdominal
muscle relaxation.
IV techniques, overall, have less of a cardiovascular depressant effect and, therefore, less frequent CV
ADRs such as hypotension. The titration of volatile agents to achieve lack of movement and/or abdominal
muscle relaxation is limited only by the development of ADRs such as hypotension. Improvements in the
ability to diagnose and monitor cardiovascular diseases and/or cardiovascular derangements like
hypotension with the use of transesophageal echocardiography in the perioperative period has allowed
anesthesiologist to maintain an adequate depth of anesthesia in severely ill patients. Thus, the traditional
use of light anesthesia techniques with the mandatory use of NMBs to ablate patient movement is no
longer required to avoid morbidity and mortality (Dryad, Background section, Cardiovascular Function &
Pathologies and Psychological sequelae&Light anaesthesia).
Volatile agents are hypnotic sedative agents that induce hypnosis and unconsciousness. They are
analgesics and induce abdominal muscle relaxation. They are difficult to use alone due to airway ADRs
such as coughing, bucking and laryngospasm. They have been associated with high rates of awareness
when used alone. As a result, supplemental small doses of narcotics are routinely used to blunt the airway
reactivity, synergize the analgesic action of the volatile agent and, thereby, enable the reduction in the
dose of volatile agent required.
Hypotension is a common ADR when MAC levels of agents are used. This is another reason that nitrous
oxide (N2O) has been used as a supplemental agent that has less cardiovascular depression than narcotics
yet still has analgesic and hypnotic sedative actions that allow a reduction in volatile agent dose. NMBs
are commonly used to ablate patient movement by many clinicians to enable lower doses of amnesic
levels of volatile agents and, therefore, less cardiovascular depressant ADRs, rather than higher doses of
agent need to maintain an unconscious patient.
IV and volatile techniques can be equally effective
IV and volatile techniques can be equally effective if they are both titrated to endpoints of surrogate
markers of patient unconsciousness: lack of movement and lack of response to command (refer to the
detailed version of reviewer#1 response). TIVA technique of propofol remifentanil is capable of being
administered in this manner due to the short duration of action of both drugs compared to other TIVA
drug combinations. Other TIVA techniques cannot be administered in this manner due to the long acting
duration of action of drugs like midazolam, alfentnail etc. hence, ceiling doses are needed in order to
extubate the patient at the end of surgery.
Titrating various anesthetic drug combinations to any endpoint that accurately tracks a patient’s level of
consciousness would render differences between anesthetic techniques unimportant because any
CARG 130 reviewer#1. defense RCT merge criteria.summary.docx
Page 3 of 7
combination of hypnotic sedative and analgesic in a high enough dose will render a patient persistently
unconscious in the setting of any surgical stimulation. This algorithm assumes that there is no ceiling dose
to enable extubating the patient and that hypotension can be managed without using NMB associated
light anesthesia techniques as part of the algorithm to treat hypotension.
High doses narcotics alone with 100% 0xygen studied in cardiac surgery in the 1970s to 80s can
inconsistently produce unconscious in patients during surgery.
What complicates this issue are inaccurate endpoints like standard clinical parameters SCPs, as defined in
the review, more so than processed EEGs and more so response to command and patient movement.
Titrating to the endpoint of lack of patient movement or lack of response to command is probably the
most accurate techniques because 60 years of case reports indicates that only a small percent of patients
are awake and do not move or respond to command. Even fewer who are in pain and or distress.
The use of ceiling doses of IV agents optimizes the ability to extubate patients at the end of a procedure
when long acting drugs are used. But, this requires the use of NMBs to ablate patient movement. NMBs
are also used as part of the light anesthetic technique used as a treatment for one of the most common
ADRs with the use of volatile agents, hypotension
Limiting the dose of IV and volatile agents for the reasons described creates unpredictable amounts of
patient movement which disrupts surgery and is unsafe and unpleasant for the patient. The mandatory
use of NMBs is required to ablate these disruptive patient movements.
The use of NMBs can be interpreted as having a positive confounding effect. That is, when the observed
association is biased away from the null hypothesis. That is, the use of NMBs creates a potentially light
anesthetic when it is used for either indication described which is associated with a higher frequency of
wakefulness/awareness (STAT507 2016).
The experts who criticized the heterogeneity of the mixed classes of drugs in these comparisons are
correct if the review is asking a narrow question regarding interventions that can reduce wakefulness and
awareness. To be consistent with their criteria, anesthesia techniques between merged studies have to
have the same drug combinations of specific drugs within hypnotic sedative and analgesic and NMB
classes of drugs. All drugs except for the randomized ones have to have the same dosing regimens, mg/kg
and/or MAC. Then, the randomized drugs vary the dosing between experimental and control. Placebo
drugs can be used in a limited fashion as was used in comparison 4, McNulty 1995 and Miller 1996 RCTs .
The randomized drugs can be used in any phase of the anesthetic: as a premedication, during induction,
maintenance or both.
More than two drugs can be randomized as was done in comparisons 3 and 4: ketamine, thiopental and
midazolam or diazepam. The problem with this approach, in regards to heterogeneity, is that there is no
consensus as to the definition of equivalent doses of different dugs within and between classes of drugs.
In addition, there is no way to quantify surgical stimulation or individual patient sensitivity to the drugs
used.
Therefore, even within narrowly defined drug combinations there are major problems with interpreting
the impact of the drugs on outcomes. In the end, the generalizability is limited to the dosing ranges in the
comparisons. And, because drug equivalents are, at best, an expert opinion, one could question the
generalizability beyond the specific drug doses. But even this is problematic because we have the
CARG 130 reviewer#1. defense RCT merge criteria.summary.docx
Page 4 of 7
confounding variables of individual patient drug sensitivity differences between patients and inability to
quantify surgical stimulation.
Given these limitations of narrowly defined questions, we chose a broader question which allowed for
greater heterogeneity of anesthetic drug groups within broader classes of drugs. That is, we classified
anesthetic techniques by the broadest classification, hypnotic sedative, analgesic and NMB drugs. within
classes. Volatile agents can achieve the endpoints of most of these classes of drugs. They have hypnotic
sedative, analgesic and abdominal relaxing actions.
The advantage of using the broader question, as compared to a narrow question, is that the broad
question results in a “comprehensive summary of the evidence and the ability to assess generalizability
of the findings across types of participants” (Higgins and Green 2011). Nonetheless, depending on the
specific aspect of the heterogeneity of either broad or narrow review, both types of questions may not be
generalizable to other settings or populations. The common limitation of both IV and volatile agent
techniques in regards to assessing their impact on wakefulness and awareness is when NMBs are added
to the drug regime because the NMBs have a confounding effect on the outcomes assessed.
The basis for this statement is as follows. IV and volatile agent techniques if titrated to unconsciousness
without the use of NMBs and ceiling dose protocols will be equivalent in their ability to render the patient
unconscious assuming that the ADRs from these techniques such as hypotension can be managed without
NMBs.
This statement is based on the assumption that the only way to differentiate between consciousness
and unconsciousness under anesthesia is the lack of response of the patient as defined by response to
command such as “open your eyes” or the lack of any type of patient movement, purposeful or nonpurposeful. We assume that non purposeful movement if not intervened with the administration of more
anesthesia will progress to purposeful movement. Some experts state that reflex movement can occur in
patients who are unconscious and conscious during anesthesia and surgery (Wilson 1981, Hung, Varvel et
al. 1992, Russell and Wang 2015)
We agree with the Cochrane guidelines that one of the justifications for asking a broader question in this
review is to make a comprehensive summary of the evidence in favor of or against processed EEGs
monitors’ and BIS, in particular, ability to decrease awareness events across a wide range of clinical
circumstances (participants, type of surgery, age groups, severity of illness etc.) (Higgins and Green 2011).
Three of the five largest RCTs in comparison 1 and 2, Avidan 2008 and 2011 and Mashour 2012, found no
difference between BIS and SCPs with ETAG/alarms/alerts. The other two large RCTs, Myles 2004 and
Zhang 2011, did find in favor of BIS. But, our subgroup analysis of both volatile and TIVA as separate
subgroups indicate that the TIVA component seemed to be the key variable based on the fact that BIS
was developed using TIVA techniques comprised of propofol based techniques. In addition, we also
demonstrated that there was methodological heterogeneity in regards to the adequate sample size
estimations (Dryad website, Potential biases in the review process.docx).
The authors in comparison 1 and 2 used both observational and RCT studies published over a 25-year
time period with widely varying awareness rates and effect size differences outcomes. This wide range
of outcome rates may have functioned as a negative confounding factor, that is, the observed association
is biased toward the null. But, we would argue that even if future studies decreased these forms of
CARG 130 reviewer#1. defense RCT merge criteria.summary.docx
Page 5 of 7
heterogeneity and confounding factors and found that BIS was significantly better than SCPs, we would
still argue that BIS tracks postoperative memory of intraoperative events and conflates awareness with
wakefulness which we consider an inadequate goal of anesthesia rather than wakefulness (an adequate
goal) (Sanders and Sleigh 2011). Furthermore, we also argue that expert interpretation of BIS ‘s ability to
reduce awareness (amnesia) is conflated with reducing uintended consciousness (wakefulness) during
surgery (Wang, Messina et al. 2012).
An examination of our classification of anesthesia techniques (Dryad, CARG 130 reviewer#1.defense RCT
merge criteria.details, charts, page 50-51) indicates that the overwhelming percentage of anesthesia
techniques within each comparison is volatile techniques for comparison 1 (80.3%/19.7%) and 2
(80.2%/19.8%), and IV is the overwhelming percentage for comparison 3 (28.4%/71.6%) and 4
(33.0%/67.0%). Comparison 5 is 100% volatile technique.
The heterogeneity of anesthetic techniques when analyzed with the denominator representing the total
number of patients merged rather than individual sample sizes is the important stratification method in
our opinion. The results of the large studies show that volatile agent techniques do not show a benefit for
BIS whereas TIVA techniques do.
We have shown that in the TIVA subgroup, the significant effect of BIS in reducing awareness becomes
non-significant when we shifted from a Peto odds ratio analysis to the analysis described in the methods
section of the review that was more appropriate for a rare or uncommon awareness outcome.
The randomization method also decreases the impact of heterogeneity and potential confounding
factors because it balances biases equally between experimental and control groups and, therefore, the
major impact of the drug combinations in both groups on the wakefulness/awareness outcome are
influenced by the randomized portion of the drug combinations.
We also argue that, based on our review of the literature, processed EEGs track consciousness more
precisely than do SCPs. We also believe that response to complex command in the paralyzed as well as
non-paralyzed state is the gold standard marker for consciousness. We concede that there is a small
percent of patients who are awake and do not respond. But, relative to the level of imprecision of SCPs
and less so processed EEGs, the false negative rate of response to command and lack of patient movement
is acceptable. This false negative outcome means that the monitor (response to command, ADMs, SCPs)
indicates patient is asleep when they are awake relative to the false negative rate of SCPs and processed
EEGs.
Given these assumptions and arguments, we would characterize that the impact of heterogeneity is less
for comparisons 1,2 and 3 relative to comparison 4. This is because of the mitigating influence of the use
of ADMs and randomization on heterogeneity described earlier. Whereas, in comparison 4, the titration
endpoints are the less precise interventions of dosing algorithms and SCPs which are mitigated by
randomization but not ADMs. Nonetheless, the large effect size difference between the two groups we
state is due to the impact of the randomized drugs and further mitigates the impact of heterogeneity and
potential confounding factors. In addition, the majority of patients were exposed to IV anesthesia. We
believe that the confounding factor is not the lack of agreement of the specific drug combinations
between the different IV techniques as the peer reviewers have stated but, rather, the requirement to
use NMBs to ablate patient movement due to the need for ceiling doses due to the long duration of
action and hence a light anesthetic technique.
CARG 130 reviewer#1. defense RCT merge criteria.summary.docx
Page 6 of 7
We feel that it is important to repeat the core issue. That is, that any class of hypnotic sedative and
analgesic drugs, if titrated to effect, will achieve hypnosis and/or unconsciousness and analgesia under
any type of surgical stimulation. The limitations of specific anesthetic techniques to consistently achieve
hypnosis and analgesia is due to the ceiling dose protocols that, in turn, require the use of NMBs to ablate
patient movement that is an ADR of light anesthesia techniques.
Comparison 5 has minimal heterogeneity and the summated results make clinical sense: high dose
volatile agents are associated with significantly less awareness than lower doses.
End summary
Key points regarding the interpretation of heterogeneity in comparisons 1 to 5
Factors that influence the impact of clinical heterogeneity
Titration of anesthesia by: Dosing regimens vs BIS target range endpoint
The use of mg/kg and MAC dosing protocols of anesthetic techniques in comparisons 4 and 5 can bias
outcomes due to the impact of potential confounding variables.
Titration of anesthesia to external endpoints in Comparison 1 to 3 rather than dosing protocols: ADM &
response to command. Hence, clinical heterogeneity’s impact on outcomes is mitigated by this protocol.
Randomization used in the five comparisons allows for the balanced distribution of potentially
confounding variables. This method had a greater impact in reducing the impact of heterogeneity in
comparisons 1 and 2 because there were no downgrades in the quality of evidence for risk of bias
whereas there was a -1 downgrade for comparisons 3 to 5.
The RCTs in (Analysis 1.1; Analysis 2.1) contained studies that used different assumptions to define high
risk awareness patients and adequate sample sizes.
The direction of effect favoring experimental or control groups, the effect size difference between
these two groups and the extent of overlap of the confidence intervals (Cis) contribute to
heterogeneity. In comparison 1 and 2 these factors were consistent with no significant difference
between ADMs and SCPs. In contrast, the impact of heterogeneity was reduced in comparisons 3 to 5, in
part due to the consistent direction in favor of the newer anesthetic medication, and higher dose of
volatile agent, a very large effect size difference between these two groups and extensive overlap of Cis.
There was significant heterogeneity between RCTs in regards to experts’ adjudication of patients’
awareness reports as “not valid” in comparison 1 and 2.
One potential confounding variable, high and not high risk awareness, showed no significant difference
between both groups frequency of awareness as described in the results section of CARG 130.
CARG 130 reviewer#1. defense RCT merge criteria.summary.docx
Page 7 of 7
Avidan, M. (2013). [Pers Comm]. Department of Anesthesiology, Washington University in Saint Louis,
School of Medicine, Campus Box 8054, 660 S. Euclid Ave., Saint Louis, MO, 63110, USA. Letter to:
Anthony Messina, 9th December 2013.
Borenstein, M., L. V. Hedges, J. P. T. Higgins and H. R. Rothstein (2009). Introduction to Meta-Analysis,
John Wiley & Sons, Ltd. ISBN 978-0-470-05724-7.
Higgins, J. P. T. and S. Green, Eds. (2011). Table 5.6.a: Some advantages and disadvantages of broad
versus narrow review questions Cochrane Handbook for Systematic Reviews of Interventions. The
Cochrane Collaboration, Version 5.1.0 [updated March 2011], www.cochrane-handbook.org.
Hung, O. R., J. R. Varvel, S. L. Shafer and D. R. Stanski (1992). "Thiopental pharmacodynamics. II.
Quantitation of clinical and electroencephalographic depth of anesthesia." Anesthesiology 77(2): 237244.
Myles, P. S. (2011). [Pers Comm], Professor/Director of Anaesthesia and Perioperative Medicine, Alfred
Hospital and Monash University, Melbourne, Australia. Peer reviewer’s assessment of CARG 130 to:
Anthony Messina, 4th June 2011
Russell, I. F. and M. Wang (2015). "The IFT (isolated forearm technique) and consciousness." Br J
Anaesth 114(3): 532.
Russo, M. W. (2007). "How to Review a Meta-analysis." Gastroenterology & Hepatology 3(8): 637-642.
Sanders, R. D. and J. W. Sleigh (2011). "Prevention of intraoperative awareness." N Engl J Med 365(21):
2033; author reply 2034-2035.
STAT507. (2016). "3.5 - Bias, Confounding and Effect Modification: Positive and negative confounding."
The Pennsylvania State University, Department of Statistics Online Programs, Penn State Eberly College
of Science, STAT 507, Epidemiological Research Methods;
https://onlinecourses.science.psu.edu/stat507/node/34.
Wang, M., A. G. Messina and I. F. Russell (2012). "The topography of awareness: a classification of intraoperative cognitive states." Anaesthesia 67(11): 1197-1201.
Wilson, M. E. (1981). "Detection of wakefulness during general anaesthesia." Br J Anaesth 53(11): 1234.