Download A clinical data warehouse-based process for refining medication

Case report
A clinical data warehouse-based process for refining
medication orders alerts
Abdelali Boussadi,1,2,3,4 Thibaut Caruba,3,5 Eric Zapletal,3 Brigitte Sabatier,3,5
Pierre Durieux,1,2,3 Patrice Degoulet1,2,3
< Additional appendices are
published online only. To view
these files please visit the
journal online (http://dx.doi.org/
10.1136/amiajnl-2012-000850).
1
Paris Descartes University
(Paris 5), Paris, France
2
INSERM, UMR S 872, Eq 20,
Paris, France
3
Georges Pompidou University
Hospital, Paris, France
4
UPMC University (Paris 6),
Paris, France
5
INSERM U 756 and LIRAES EA
4470, Paris, France
Correspondence to
Abdelali Boussadi, Département
d’Informatique Hospitalière
(DIH), Hôpital Européen Georges
Pompidou, 20 Rue Leblanc,
75908 Paris Cedex 15, France;
[email protected]
Received 13 January 2012
Accepted 23 March 2012
Published Online First
20 April 2012
ABSTRACT
The objective of this case report is to evaluate the use of
a clinical data warehouse coupled with a clinical
information system to test and refine alerts for
medication orders control before they were fully
implemented. A clinical decision rule refinement process
was used to assess alerts. The criteria assessed were
the frequencies of alerts for initial prescriptions of 10
medications whose dosage levels depend on renal
function thresholds. In the first iteration of the process,
the frequency of the ‘exceeds maximum daily dose’
alerts was 7.10% (617/8692), while that of the ‘under
dose’ alerts was 3.14% (273/8692). Indicators were
presented to the experts. During the different iterations
of the process, 45 (16.07%) decision rules were
removed, 105 (37.5%) were changed and 136 new rules
were introduced. Extensive retrospective analysis of
physicians’ medication orders stored in a clinical data
warehouse facilitates alert optimization toward the goal
of maximizing the safety of the patient and minimizing
overridden alerts.
INTRODUCTION
Implementing a clinical decision rule involves three
main steps: creating the rule, testing it, and
assessing its impact on clinician behavior.1 In most
rule-based decision support systems studies,
assessing the impact of the clinical decision rules
development step is over-emphasized to the detriment of the rule testing development step.2 3
Testing clinical decision rules after they have
been fully implemented can be resource intensive
and time consuming.4 First, operational databases
relying upon computerized provider order entry
(CPOE) and electronic medical record components
are difficult to query for testing and validating the
implemented clinical decision rules.5 The multiplicity of relational tables, which are optimized to
facilitate patient management, impedes the efficiency of the statistical queries necessary for validating the rules. The CPOE and electronic medical
record components architecture do not clearly
separate the business logic from the technical logic
of implementation. As a result, it is difficult to
modify the rules based on the results of testing.
Second, testing clinical decision rules in a production system is computationally intensive and
requires specific expertise during alert development
as described by Oppenheim et al.4
An alternative is to use a large ‘retrospective’
chart review.3 The present study used a clinical
data warehouse (CDW) to test and refine (new)
782
clinical decision rules for medication order alerts
regarding drug dosages for inpatients with renal
insufficiency before the rules were fully implemented and/or reintroduced.
METHODS
Study site and settings
Hôpital Européen Georges Pompidou (HEGP) is
a teaching hospital with 24 clinical departments
and 795 beds. The HEGP clinical information
system integrates an electronic health record with
a CPOE (DxCare, MEDASYS).6 A CDW based on
the i2b2 framework was integrated into the HEGP
clinical information system in 2009.7
Data with the electronic health record database
and the CPOE database are automatically reflected
into the CDW database. The clinical data used in
this study to assess decision rules were derived
from medication orders and biological investigations ordered between January 2005 and March
2011 at the HEGP. The greatest benefit of using the
i2b2 in relation to our objectives was the simplicity
of the i2b2 star schema which includes only five
tables.7
Medication list selection
After having analyzed existing guidelines,8 9
a local medical expert panel defined the dosage
adjustments according to the estimated glomerular filtration rate (eGFR) of patients for the
medications considered (table 1). The eGFR values
were determined using the revised 4-components
of an equation from the Modification of Diet
in Renal Disease (MDRD) study,10 with the
exception of the ethnic component, which has not
been validated in a European context. Required
dosage adjustments were based on the level of
kidney function impairment. For each medication
and for each eGFR category, the expert panel
determined:
< The ‘minimum dosage’ per 24 h as the lowest
medication dosage irrespective of indication;
< The ‘maximum dosage’ per 24 h as the highest
medication dosage irrespective of indication.
Clinical decision rule refinement
We developed a first set of clinical decision rules to
monitor prescriptions for the medications listed in
table 1. Clinical decision rules were represented as
logical ‘If <Conditions>.Then <Actions>.Else’
statements to be followed to reach a conclusion.2 11
A rule is acted upon, that is, the <action> part is
performed, when the <condition> part is true. A
patient’s eGFR value is an example of a decision
J Am Med Inform Assoc 2012;19:782e785. doi:10.1136/amiajnl-2012-000850
Case report
Table 1
Comparison between alert prevalences before and after the first iteration of the clinical decision rules refinement process
Medications
Patients
Number of
prescriptions
analyzed
Amoxicillin and
clavulanic acid
Amoxicillin and
clavulanic acideoral
Amoxicillin and
clavulanic acide
IVe2000/200
Amoxicillin and
clavulanic
acideIVe1000/200
Cefotaxim
Ethambutol
Isoniazid
Metronidazole
Acyclovir
Amoxicillin
Metformin
Tramadol
Valacyclovir
2801
3253
1965
2273
35
42
801
938
363
67
87
531
93
1284
913
322
266
428
109
173
730
134
2027
1131
351
356
First iteration
Max daily dosage
alerts frequency
Under-dosage
alerts frequency
Second iteration
Total alerts
307 (9.44%)
90 (2.77%)
397 (12.20%)
Max daily dosage
alerts frequency
Under-dosage
alerts frequency
Total alerts
316 (13.90%)
78 (3.43%)
394 (17.33%)
3 (7.14%)
3 (7.14%)
8 (0.85%)
19 (2.03%)
0 (0%)
11 (1.17%)
17
13
5
2
4
54
194
21
0
(3.97%)
(11.93%)
(2.89%)
(0.27%)
(2.99%)
(2.66%)
(17.15%)
(5.98%)
(0%)
20
11
92
7
4
8
33
6
2
(4.67%)
(10.09%)
(53.18%)
(0.96%)
(2.99%)
(0.39%)
(2.92%)
(1.71%)
(0.56%)
rule condition. One or more eGFR values can be estimated for
a patient during his hospital stay. The clinical decision rules have
been designed to use the eGFR value which has the closest
entered date/time attribute to the date/time attribute of the
corresponding medication order.
Figure 1 describes the different steps of the clinical decision
rule test and refinement process. We submitted these rules to the
refinement process to determine the nature of the resulting
changes which could impact the decision rules and the alert
incidence.
RESULTS
We implemented 280 clinical decision rules based on a business rule management system (BRMS) and connected them to
the CDW database to retrospectively control orders of 10
medications (table 1). The details of the alert system architec-
37
24
97
9
8
62
229
27
2
(8.64%)
(22.02%)
(56.07%)
(1.23%)
(5.97%)
(3.06%)
(20.25%)
(7.69%)
(0.56%)
8
18
12
32
(1.87%)
(16.51%)
(6.94%)
(4.38%)
13
1
11
12
(3.04%)
(0.92%)
(6.36%)
(1.64%)
21
19
23
44
(4.91%)
(17.43%)
(13.29%)
(6.03%)
ture are described in online appendix B and were reported
previously.2
Clinical decision rule refinement
Changes to the clinical decision rule condition value
The alert system analyzed 8692 drug prescriptions ordered
between 2008 and 2011 for 6727 different patients in 17
different medical departments. The alert system triggered 892
alerts (10.26%), 617 (7.10%) exceeds max daily dose alerts and
273 (3.14%) under-dosage alerts (table 1). The indicators
summarized in table 1 were presented to the HEGP expert panel
(step 4 in figure 1). Based on the results of the first iteration of
the refinement process, the panel decided to modify the medication dosage rates for four medications: cefotaxim, ethambutol,
isoniazid, and metronidazole (see online appendix A for the
overall modifications). In this case, steps 15 were followed in
the refinement process.
Figure 1 The different steps of the clinical decision rule refinement process. CDW, clinical data warehouse; CPOE, computerized provider order entry;
EMR, electronic medical record; HEGP, Hôpital Européen Georges Pompidou.
J Am Med Inform Assoc 2012;19:782e785. doi:10.1136/amiajnl-2012-000850
783
Case report
Antibiotic under-dosage presents a serious risk of exposing the
patient to resistant bacteria and thus worsening the infection.12
13
Pharmacists were notified of the under-dosage alert rates
associated with the isoniazid and ethambutol prescriptions
checked by the alert system (table 1). The pharmacists then
discussed these rates with the HEGP physicians in charge of
tuberculosis treatment. The physicians informed the pharmacists
that the average weight of the patients associated with these
prescription alerts was between 50 and 60 kg (online appendix A).
The clinical decision rules of the first iteration test case had been
configured to check the isoniazid and ethambutol prescriptions of
patients weighing at least >70 kg (online appendix A). In this first
test case iteration, a possible explanation for the under-dosage
alert rate for ethambutol prescriptions (10.09%) and isoniazid
prescriptions (53.18%) (see table 1) was that the pharmacists had
initially overestimated the weights of the patients associated
with these prescriptions. This hypothesis was confirmed in the
second test case iteration, in which the under-dosage alert rates
decreased for the checked prescriptions of isoniazid and ethambutol to 6.36% and 0.92%, respectively (table 1).
Similarly, the cefotaxim high under-dosage alert rate observed
in the first test case (table 1) emphasized the relevance of the
decision rules related to cefotaxim prescription. Cefotaxim
dosages (online appendix A) configured by the pharmacists in
the first clinical decision rules were derived from Drug Prescribing
in Renal Failure.9 The medication dosages recommended by this
resource are higher than those listed in the Guidelines for
Prescription in Renal Disease (GPR) handbook.8 Indeed, according
to Drug Prescribing in Renal Failure, the recommended daily
cefotaxim dosages are between 2000 and 4000 mg for patients
with eGFRs between 20 and 50 ml/min/1.73 m2 and between
1000 and 2000 mg for patients with eGFRs of <10 ml/min. In
the GPR handbook, the listed minimum daily dosages are
1500 mg for patients with eGFRs between 15 and 30 ml/min/
1.73 m2 and 750 mg for those with an eGFR of <15 ml/min/
1.73 m2. French physicians often prefer to consult the GPR
handbook for antibiotic dosage information. For this reason, the
pharmacists reviewed and adjusted the initial clinical decision
rules according to the GPR handbook recommendations.8
Alerts for high and low doses of metronidazole increased after
the second iteration of the refinement process (from 0.27% to
4.38%, and from 0.96% to 1.64%, respectively; see table 1).
Metronidazol dosages (online appendix A) configured by the
pharmacists in the first iteration of the refinement process were
also derived from Drug Prescribing in Renal Failure.9 Similarly to
the case of cefotaxim, isoniazid, and ethambutol, and according
to the daily practice of French physicians, pharmacists reconfigured the clinical decision rules with the dosage recommendations of the GPR handbook.8 These two guidelines differ only
in the maximum daily dose recommended when the eGFR is
<10 ml/min/1.73 m2: the GPR handbook recommends 750 mg
and Drug Prescribing in Renal Failure recommends 1500 mg
(online appendix A). Thus, the lower threshold for the
maximum daily dose of metronidazole resulted in a substantially higher number of ‘exceeds maximum dosage’ alerts for the
second configuration.
Adding new clinical information
In the first iteration of the refinement process, pharmacists
informed the alert system administrator that the clinical alert
recommendations for amoxicillin and clavulanic acid did not
distinguish the branded names prescribed for oral administration
from the branded names prescribed for intravenous administration (online appendix A).
784
A second iteration of the refinement process was performed
and steps ‘a’ and ‘b’ (figure 1) were followed to include the
medication administration route in the clinical decision rules as
a new condition part.
New indicators were presented to the HEGP expert panel
(table 1). Pharmacists decided in the second iteration to distinguish between the different amoxicillin and clavulanic acid
forms for intravenous administration (amoxicillin and clavulanic
acideIVe2000/200 and amoxicillin and clavulanic acideIVe2000/100). Table 1 compares the prevalence of alerts before
and after adding these new criteria to the clinical decision rules
for amoxicillin and clavulanic acid prescriptions.
Number of decision rules affected during the experiments
Table 2 reports the number of clinical decision rules removed,
modified, or added in response to the policy changes made by the
pharmacists during the execution of the refinement process.
DISCUSSION
During the different iterations of the decision rule refinement
process (figure 1), 45 (16.07%) decision rules were removed, 105
(37.5%) were changed, and 136 new rules were introduced (table
2). Executing a test development phase before proceeding with
a prospective study allowed a priori control of the performance
indicators of the alerts, which can lower alert fatigue syndrome.
Some of the problems related to alert refinements that arise in
prospective studies can be addressed and eliminated during a test
development phase. This allows alerts to be improved before
prospective assessment and full integration into the hospital
information system, leaving study investigators with more time
to focus on other problems related to clinical alert monitoring.
Both interruptive and non-interruptive alerts can be useful in
daily clinical practice,14 although there is some difficulty in
determining the threshold between a non-interruptive alert and
an interruptive alert. Our approach could facilitate the definition
of these thresholds by pre-analyzing the impact of clinical
decision rules. Hence, experts can determine whether certain
recommendations are less critical than others and maintain
these in a non-interruptive mode. Our approach could be used to
test and refine other decision rules templates for medication
orders control,15 for example, decision rules for hepatic function
medication orders control,16 drugedrug interactions,17 etc. In
addition, this method is currently used to design and test decision rules for clinical trial patient recruitment,18 19 thus allowing
our method to be tested in a different setting.
Table 2
Clinical decision rules affected by the refinement processes
Initial number
of rules
First experiment
Metronidazole
Cefotaxim
Isoniazid
Ethambutol
Acyclovir
Amoxicillin
Metformin
Tramadol
Valacyclovir
Second experiment
Amoxicillin and
clavulanic acid
Total
Rules
removed
Rules
changed
Rules
added
24
24
30
27
40
16
18
16
40
0
0
0
0
0
0
0
0
0
24
24
30
27
0
0
0
0
0
0
0
0
0
0
0
0
0
0
45
45
0
136
280
45 (16.07%)
105 (37.5%)
136
J Am Med Inform Assoc 2012;19:782e785. doi:10.1136/amiajnl-2012-000850
Case report
Our results show that using a CDW as an experimental
platform to test clinical decision rules before they are fully
implemented allows medication dose thresholds to be easily
adjusted according to experts’ opinions. This experimental
platform also allows assessment of the impact of the modifications on alert frequencies and possibly the false-positive rate of
the alerts. Such testing can result in radical modifications to
clinical decision rules, such as the introduction or suppression of
new facts in the condition part of the clinical decision rules.
In the first iterations of the clinical decision rule design and
refinement process, decision rules were designed to control several
generic drug names.15 After several iterations of the process, the
number of conditions in the condition part of the rules increased
and so the number of decision rules also increased. The case of
amoxicillin and clavulanic acid is a good example (tables 1 and 2).
In the first iteration of the process, the decision rules designed to
control the amoxicillin and clavulanic acid medication orders did
not consider the administration route as a condition. After the
administration route was introduced as a condition, the number
of rules increased from 45 to 136 (table 2).
Regardless of the clinical decision rule software design process
and the degree of end-user involvement in this process, CDWbased retrospective decision rule testing enables the identification and correction of design errors and omissions in the clinical
data used by the rules, supporting an ‘agile’ process.2 Nonetheless, careful testing before implementation does not eliminate
the need for regular monitoring of the performance of clinical
decision rules after implementation.
Acknowledgments We thank IBM-ILOG for providing us with an evaluation version
of IBM WebSphere ILOG JRules.
Contributors AB made a substantial contribution to the conception and design of this
study, acquisition, analysis and interpretation of data, and drafting of the article. TC,
EZ, BS, and PDX made substantial contributions to the acquisition of data, analysis and
interpretation of data, and critically revising the article. PD as PhD mentor of AB, made
substantial contributions to framing the research objectives, analysis and
interpretation of data, and critically revising the article.
Funding This study and AB were supported by a grant from the Programme de
recherche en qualité hospitalière (PREQHOSdPHRQ 1034 SADPM), The French
Ministry of Health and the Electronic Health Record For Clinical Research (EHR4CR)
project, Grant Agreement no. 115189.
Correction notice This article has been corrected since it was published Online First.
A mistake in the second paragraph of the Methods; study site and settings section
has been corrected and the last sentence of the contributors section has been
amended.
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.
REFERENCES
1.
2.
3.
4.
5.
Limitations and perspectives
The decision rules we tested were only for adjusting the dosages
of medications whose dose depends on the level of renal function.
To date, we have not yet implemented and tested this refinement
process for developing and maintaining more advanced decision
rules, such as those involving complex pharmaco-dynamic
models or sequential drug administration. However, our
approach could be extended to such situations in principle.
Retrospective testing of clinical decision rules establishes
a ‘partial’ external validity of these rules.20 However, the rules
must be further validated by prospective testing.1 An evaluation
study assessing the diagnostic performance of a clinical decision
rule-based alert system developed using the proposed framework, will allow complete validation of clinical decision rules.
The results of such a study will allow us to determine how our
rate of false-positive alerts compares to that of other systems
that did not use a clinical decision rules refinement step during
the development phase of their alert system.
CONCLUSION
The most common reason for overriding alerts, which can
impair patient safety, is alert fatigue.21 One way to reduce alert
fatigue is to reduce the frequency of false-positive alerts or alerts
not accepted by clinicians.22 This recommendation can be
difficult to implement because it requires definition of the
correct balance among the requirements of the clinicians, the
hospital, and the safety of the patients.23 Testing decision rules
before they are fully implemented may help determine the right
balance. This can be accomplished by performing several test
iterations on data downloaded from the CPOE or from the
pharmacy systems of the clinical site for which the alert system
is intended.24 The method described in this study can facilitate
clinical alert optimization to maximize the safety of the patient
and to minimize overridden clinical alerts.
J Am Med Inform Assoc 2012;19:782e785. doi:10.1136/amiajnl-2012-000850
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
McGinn TG, Guyatt GH, Wyer PC, et al. Users’ guides to the medical literature: XXII:
how to use articles about clinical decision rules. Evidence-Based Medicine Working
Group. JAMA 2000;284:79e84.
Boussadi A, Bousquet C, Sabatier B, et al. A business rules design framework for
a pharmaceutical validation and alert system. Methods Inf Med 2011;50:36e50.
Shapiro SE. Guidelines for developing and testing clinical decision rules. Western
Journal of Nursing research. 2006;28:244e53.
Oppenheim MI, Mintz RJ, Boyer AG, et al. Design of a clinical alert system to
facilitate development, testing, maintenance, and user-specific notification. Proc
AMIA Symp 2000:630e4.
Wilcox A, Hripcsak G, Chen C. Creating an environment for linking knowledge-based
systems to a clinical database: a suite of tools. Proc AMIA Annu Fall Symp 1997:303e7.
Degoulet P, Marin L, Lavril M, et al. The HEGP component-based clinical information
system. Int J Med Inform 2003;69:115e26.
Zapletal E, Rodon N, Grabar N, et al. Methodology of integration of a clinical data
warehouse with a clinical information system: the HEGP case. Stud Health Technol
Inform 2010;160:193e7.
GPR. Guidelines for Prescription in Renal Disease. http://www.sitegpr.com/page.
php?pg¼gpr (accessed 18 Feb 2012).
Aronoff GR, Bennett WM, Berns JS, et al. Drug Prescribing in Renal Failure: Dosing
Guidelines for Adults. 5th edn. Philadelphia: American College of Physicians, 2007.
Sellier E, Colombet I, Sabatier B, et al. Effect of alerts for drug dosage adjustment in
inpatients with renal insufficiency. J Am Med Inform Assoc 2009;16:203e10.
Greenes RA, ed. Clinical decision support: the road ahead. Burlington, MA:
Academic Press, 2007.
Guillot E, Sermet I, Ferroni A, et al. Suboptimal ciprofloxacin dosing as a potential
cause of decreased Pseudomonas aeruginosa susceptibility in children with cystic
fibrosis. Pharmacotherapy 2010;30:1252e8.
Guillemot D, Carbon C, Balkau B, et al. Low dosage and long treatment duration of
beta-lactam: risk factors for carriage of penicillin-resistant Streptococcus
pneumoniae. JAMA 1998;279:365e70.
Lo HG, Matheny ME, Seger DL, et al. Impact of non-interruptive medication laboratory
monitoring alerts in ambulatory care. J Am Med Inform Assoc 2009;16:66e71.
Boussadi A, Bousquet C, Sabatier B, et al. Specification of business rules for the
development of hospital alarm system: application to the pharmaceutical validation.
Stud Health Technol Inform 2008;136:145e50.
Lesar TS, Briceland L, Stein DS. Factors related to errors in medication prescribing.
JAMA 1997;277:312e17.
Zwart-van Rijkom JEF, Uijtendaal EV, ten Berg MJ, et al. Frequency and nature of
drug-drug interactions in a Dutch university hospital. Br J Clin Pharmacol
2009;68:187e93.
Embi PJ, Jain A, Clark J, et al. Effect of a clinical trial alert system on physician
participation in trial recruitment. Arch Intern Med 2005;165:2272e7.
EHR4CR. Electronic Health Records for Clinical Research. http://www.ehr4cr.eu/
(accessed 18 Feb 2012).
Kuperman GJ, Fiskio JM, Karson A. A process to maintain the quality of
a computerized knowledge base. Proc AMIA Symp 1999:87e91.
van der Sijs H, Aarts J, Vulto A, et al. Overriding of drug safety alerts in
computerized physician order entry. J Am Med Inform Assoc 2006;13:138e47.
Grissinger M. Warning! Don’t Miss Important Computer Alerts. Pharmacy and
Therapeutics 2010;35:368e72.
Peitzman L. Reducing medication alert fatigue to Improve physician Adoption of
computerized physician order entry. Pain Medecine News. 2010. http://www.
painmedicinenews.com/ViewArticle.aspx?d_id¼83&a_id¼15592
Cash JJ. Alert fatigue. Am J health Syst Pharm 2009;66:2098e101.
785
PAGE fraction trail=4