Download Assessment of the EuroSCORE risk scoring system for patients

[Downloaded free from http://www.ijccm.org on Sunday, October 11, 2015, IP: 54.152.69.150]
576
Research Article
Assessment of the EuroSCORE risk scoring system
for patients undergoing coronary artery bypass graft
surgery in a group of Iranian patients
Abstract
Hamidreza Jamaati, Arvin Najafi1, Farima Kahe1, Zahra Karimi1, Zarghamhossein Ahmadi,
Mohammadreza Bolursaz2, Mohammadreza Masjedi, Aliakbar Velayati2,
Seied Mohammadreza Hashemian
Background and Aims: Previous studies around the world indicated validity and accuracy
of European System for Cardiac Operative Risk Evaluation (EuroSCORE) risk scoring system
we evaluated the EuroSCORE risk scoring system for patients undergoing coronary artery
bypass graft (CABG) surgery in a group of Iranian patients. Materials and Methods: In this
cohort 2220 patients more than 18 years, who were performed CABG surgery in Massih
Daneshvari Hospital, from January 2004 to March 2010 were recruited. Predicted mortality
risk scores were calculated using logistic EuroSCORE and Acute Physiology and Chronic
Health Evaluation II (APACHE II) and compared with observed mortality. Calibration was
measured by the Hosmer–Lemeshow (HL) test and discrimination by using the receiver
operating characteristic (ROC) curve area. Results: Of the 2220 patients, in hospital deaths
occurred in 270 patients (mortality rate of 12.2%).The accuracy of mortality prediction in
the logistic EuroSCORE and APACHE II model was 89.1%; in the local EuroSCORE (logistic)
was 91.89%; and in the local EuroSCORE support vector machines (SVM) was 98.6%.The
area under curve for ROC curve, was 0.724 (95% confidence interval [CI]: 0.57–0.88) for
logistic EuroSCORE; 0.836 (95% CI: 0.731–0.942) for local EuroSCORE (logistic); 0.978 (95%
CI: 0.937–1) for Local EuroSCORE (SVM); and 0.832 (95% CI: 0.723–0.941) for APACHE
II model. The HL test showed good calibration for the local EuroSCORE (SVM), APACHE
II model and local EuroSCORE (logistic) (P = 0.823, P = 0.748 and P = 0.06 respectively);
but there was a significant difference between expected and observed mortality according
to EuroSCORE model (P = 0.033). Conclusion: We detected logistic EuroSCORE risk
model is not applicable on Iranian patients undergoing CABG surgery.
Access this article online
Website: www.ijccm.org
DOI: 10.4103/0972-5229.167033
Quick Response Code:
Keywords: Coronary artery bypass graft, European System for Cardiac Operative Risk
Evaluation, mortality, risk stratification, scoring system, validity
Introduction
In cardiac surgery the precise estimate models of
risks is most important for surgeons and patients.[1]
From:
Chronic Respiratory Disease Research Center, 2Pediatric Respiratory Disease
Research Center, National Research Institute of Tuberculosis and Lung
Diseases, Shahid Beheshti University of Medical Sciences, Tehran,
1
Tehran University of Medical Sciences, Tehran, Iran
Correspondence:
Dr. Seied Mohammadreza Hashemian, Chronic Respiratory Disease
Research Center, National Research Institute of Tuberculosis and Lung
Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran.
E‑mail: [email protected]
This technique provides a useful tool for surgeons to
make a correct decision whether coronary artery bypass
graft (CABG) is a suitable intervention, which patients
This is an open access article distributed under the terms of the Creative
Commons Attribution‑NonCommercial‑ShareAlike 3.0 License, which allows
others to remix, tweak, and build upon the work non‑commercially, as long as the
author is credited and the new creations are licensed under the identical terms.
For reprints contact: [email protected]
How to cite this article: Jamaati H, Najafi A, Kahe F, Karimi Z, Ahmadi Z, Bolursaz
M, Masjedi M, Velayati A, Hashemian SM. Assessment of the EuroSCORE risk scoring
system for patients undergoing coronary artery bypass graft surgery in a group of
Iranian patients. Indian J Crit Care Med 2015;19:576-9.
© 2015 Indian Journal of Critical Care Medicine | Published by Wolters Kluwer - Medknow
Page no. 14
[Downloaded free from http://www.ijccm.org on Sunday, October 11, 2015, IP: 54.152.69.150]
Indian Journal of Critical Care Medicine October 2015 Vol 19 Issue 10
577
should be carefully managed and monitored due to
adverse outcomes of the operation; [2] furthermore
the surgeons can properly inform patients and take
preoperative consent.[2] Moreover risk models are a
helpful tool to detect differences in risk profiles and to
organize the maximum use of health care resources.[3,4]
To predict the risks of CABG surgery several scoring
systems were developed by scientists.[5‑8] The Parsonnet
system was introduced in the USA[5] the European System
for Cardiac Operative Risk Evaluation (EuroSCORE)
was established, based on European population data[6]
the American College of Cardiology/American Heart
Association presented a scoring system for prediction
of outcomes after isolated CABG in America [7] and
Society of Cardiothoracic Surgeons of Great Britain and
Ireland suggested the UK CABG Bayes model for UK
patients undergoing CABG.[8] The EuroSCORE is the
most prevalent referenced and was established during
1995–1999 to present a risk model in adult European
patients under cardiac surgery.[6,9] It was formed to
predict operative mortality, the length of stay in the
Intensive Care Units (ICU)[10] complications[11,12] and costs
in cardiac surgery.[13,14] EuroSCORE risk models consisted
of 17 independent factors and evaluate mortality during
30 days after cardiac surgery and it is widely accepted in
Europe and elsewhere.[14] This study aimed to assess the
EuroSCORE risk scoring system in patients undergoing
CABG surgery in a group of Iranian patients.
Materials and Methods
As an observational, prospective cohort study
2220 patients more than 18 years undergoing CABG
surgery who admitted in ICU, from January 2004 to March
2010 were recruited. The study was approved by the
Ethical Committee of our university. Moreover the study
procedure was explained for all patients and informed
written consents were taken. To assess the EuroSCORE
and Acute Physiology and Chronic Health Evaluation
II (APACHE II) the patients were fully examined before
operation moreover demographic data such as age, sex,
race, patient risk factors and comorbidities related to
short‑term and long‑term outcomes, patient disposition,
and complications of care were extracted from medical
records of patients. Duration of ICU and hospital stay
and the possibility of death (main end point of the study)
in ICU admission 1st day based on logistic EuroSCORE
and APACHE II score were assessed. The coefficients
of EuroSCORE variables were re‑estimated on this
group as two local EuroSCORE models, with logistic
regression and support vector machines (SVM) and the
discriminative power and calibration of these models were
compared. In order to assess the EuroSCORE calibration,
the Hosmer–Lemeshow (HL) test of goodness of fit was
applied.[15] The accuracy of the model was calculated by
the receiver operating characteristic (ROC) curve (ROC
curve), designed for sensibility (accurate death prediction)
and specificity (accurate prediction of survival), analyzed
for each value of each score studied. The area under
curve (AUC) was defined as the area under the ROC curve.
The 0.5 AUC value indicates a random distinguishing of
the patients being alive and dead. An increasing value of
AUC from 0.5 toward 1.0 shows increasing distinctiveness
and better discrimination of the patients’ status. AUC
values were calculated for logistic EuroSCORE and
APACHE II and logistic regression and SVM models to test
discrimination and to describe performance and accuracy.
Categorical variables are displayed as numbers and/or
percentages, and continuous variables are displayed as
mean ± standard deviation. The results were presented
with 95% confidence intervals (CIs). A two‑sided P < 0.05
was considered as statistical significance. The statistical
program employed was SPSS (Statistical Package for the
Social Sciences) – version 18.0 for windows.
Results
A total of 2220 patients undergoing cardiac surgery
between January 2004 and March 2010 were evaluated.
The clinical characteristics of our patients and the
EuroSCORE are presented in Table 1. There were
significant differences between our patient group and
European cardiac surgical populations. In comparison
the patients in current survey were younger than
the European patients. Moreover, our patients were
more prone to have unstable angina, moderate left
Table 1: Demonstrates comparative prevalence of risk
factors in Iranian and European population
Risk factor
n
Mean age (year)
Female
Chronic pulmonary disease
Extracardiac arthropathy
Neurological disease
Previous cardiac surgery
Serum creatinine >200 µmol/L
Active endocarditis
Critical preoperative state
Unstable angina
LV dysfunction moderate or LVEF
30-50%
LV dysfunction poor or LVEF <30
Recent myocardial infarct
Pulmonary hypertension
Emergency surgery
Other than isolated CABG
Surgery on thoracic aorta
Postinfarct septal rupture
Iranian
prevalence (%)
EuroSCORE
prevalence (%)
740
57
49
11.5
2.7
4.1
12.2
6.8
2.7
4.1
79.7
56.1
19,030
62.5
27.8
3.9
11.3
1.4
7.3
1.8
1.1
4.1
8
25.6
7.4
6.8
8.1
12.8
33.8
1.4
0.7
5.8
9.7
2
4.9
36.4
2.4
0.2
LV: Left ventricular; LVEF: Left ventricular ejection fraction; CABG: Coronary artery
bypass graft
Page no. 15
[Downloaded free from http://www.ijccm.org on Sunday, October 11, 2015, IP: 54.152.69.150]
578
Indian Journal of Critical Care Medicine October 2015 Vol 19 Issue 10
ventricular function, critical preoperative state, active
endocarditis, chronic pulmonary disease compared
to European population. Fewer patients have recent
myocardial infarction, extra cardiac arteriopathy, and
surgery on thoracic aorta. The prevalence of female
sex, neurological disease, serum creatinine >0.2 mol/l,
pulmonary hypertension, emergency surgery, previous
cardiac surgery, and surgery other than isolated CABG
surgery and postinfarct septal rupture was higher
than European population. Of the 2220 patients, in
hospital deaths occurred in 270 patients (mortality rate
of 12.2%). The accuracy of mortality prediction in the
logistic EuroSCORE and APACHE II model was 89.1%;
in the local EuroSCORE (logistic) was 91.89%; and in
the local EuroSCORE (SVM) was 98.6%. The AUC for
ROC (ROCs) curve, was 0.724 (95% CI: 0.57–0.88) for
logistic EuroSCORE; 0.836 (95% CI: 0.731–0.942) for
local EuroSCORE (logistic); 0.978 (95% CI: 0.937–1)
for local EuroSCORE (SVM); and 0.832 (95% CI:
0.723–0.941) for APACHE II model [Figure 1]. The HL
goodness of fit test showed good calibration for the
local EuroSCORE (SVM), APACHE II model and local
EuroSCORE (logistic) (P = 0.823, P = 0.748 and P = 0.06
respectively); but there was a significant difference
between expected and observed mortality according
to EuroSCORE model (P = 0.033). In order to models
calibration HL test was applied [Figure 2].
Discussion
It is well‑established in some studies that EuroSCORE
is discriminative and accurate model predictions of
operative mortality,[16] however, previous study in Iran
revealed it is not valid in Iranian population.[17] The
aim of this experience was to evaluate the EuroSCORE
in CABG surgery in Iranian patients. Therefore we
evaluated the validity of the logistic EuroSCORE model
Figure 1: The calibration plot for comparing four different models
in Iranian patients undergoing cardiac surgery by
testing its calibration power and discrimination power.
Discrimination power was assessed by calculating area
under ROC curve which was 0.724 (95% CI: 0.57–0.88)
for logistic EuroSCORE. The discriminatory power
was considered excellent if AUC was >0.80, very good
if >0.75, and good if >0.70.[18] In summary the results
of our survey indicated that logistic EuroSCORE risk
model is not accurate for predicting mortality at all risk
subgroups in Iranian patients. Moreover we detected
considerable differences in patient demographics
between the Iranian and 1995 EuroSCORE data sets. In
line with our experience Yap et al. reported that the model
was not valid in Australia because of different patient
characteristics and different prevalence of risk factors.
They revealed a significant low mortality in contrast to
predicted mortality by means of EuroSCORE data sets.[19]
Harmoniously Bhatti et al. prospectively evaluated British
data containing 9995 patients from “North West Quality
Improvement Program in Cardiac Interventions.” They
reported that the discrimination power of the logistic
EuroSCORE was good with ROC curve area of 0.79 for
all types of cardiac surgery, but overestimated in‑hospital
mortality. [20] In accordance with Bhatti et al. study
D’Errigo et al. observed 30,610 isolated CABG surgeries
and found that their observed mortality was significantly
lower than the predicted mortality according to the
logistic EuroSCORE.[21] These studies were not surprising
because the EuroSCORE scoring system was introduced
15 years ago. Because to date, there have been wonderful
advancements in surgical methods, anesthetic and
postoperative intensive care quality. Consequently, all
of these have led to better surgical results and reduced
mortality. On the other hand, although the EuroSCORE
database is UpToDate and several studies indicated that,
the EuroSCORE model has been validated on patients
Figure 2: The receiver operating characteristic curve for comparison of
discriminative power of different models
Page no. 16
[Downloaded free from http://www.ijccm.org on Sunday, October 11, 2015, IP: 54.152.69.150]
Indian Journal of Critical Care Medicine October 2015 Vol 19 Issue 10
579
in Japan [22] and North American. [23] However, it is
derived from a cross‑section of contemporary European
cardiac surgery. So, it may appropriate database for
the construction of a risk evaluation scoring system for
use in Europe and it may provide conflicting results in
CABG patients in other regions like Iran (West of Asia).
In line with us a study in Thailand in Southeast of Asia
showed EuroSCORE is not valid[24] and in Netherland in
north of Europe van Straten et al. indicated additive and
logistic Euroscore are overestimating mortality rate.[25]
Additionally, important developments in CABG surgery
methods and postoperative care after the creation of the
risk scoring systems also should be considered.
8.
Conclusion
13.
We detected logistic EuroSCORE risk model is not
applicable on Iranian patients’ undergoing CABG
surgery. However, larger studies are required to confirm
results reported here. Moreover due to demographic
difference between Iranian and European patients,
creation a new specific and local risk stratification system
is essential for new experiences in Iranian patients.
9.
10.
11.
12.
14.
15.
16.
Financial support and sponsorship
Nil.
17.
Conflicts of interest
There are no conflicts of interest.
18.
19.
References
1.
2.
3.
4.
5.
6.
7.
Ferraris VA, Ferraris SP. Risk stratification and comorbidity. In:
Cohn LH, Edmunds LH Jr, editors. Cardiac Surgery in the Adult.
2nd ed. New York: McGraw‑Hill; 2003. p. 187‑224.
Hammermeister KE. Risk, predicting outcomes, and improving care.
Circulation 1995;91:899‑900.
Nashef SA, Roques F, Michel P, Cortina J, Faichney A, Gams E, et al.
Coronary surgery in Europe: Comparison of the national subsets of the
European system for cardiac operative risk evaluation database. Eur
J Cardiothorac Surg 2000;17:396‑9.
Smith PK, Smith LR, Muhlbaier LH. Risk stratification for
adverse economic outcomes in cardiac surgery. Ann Thorac Surg
1997;64 6 Suppl: S61‑3.
Parsonnet V, Dean D, Bernstein AD. A method of uniform stratification
of risk for evaluating the results of surgery in acquired adult heart
disease. Circulation 1989;79 (6 Pt 2):I3‑12.
Nashef SA, Roques F, Michel P, Gauducheau E, Lemeshow S, Salamon R.
European system for cardiac operative risk evaluation (EuroSCORE).
Eur J Cardiothorac Surg 1999;16:9‑13.
Eagle KA, Guyton RA, Davidoff R, Ewy GA, Fonger J, Gardner TJ, et al.
ACC/AHA guidelines for coronary artery bypass graft surgery:
Executive summary and recommendations: A report of the American
College of Cardiology/American Heart Association Task Force on
Practice Guidelines (Committee to revise the 1991 guidelines for
coronary artery bypass graft surgery). Circulation 1999;100:1464‑80.
20.
21.
22.
23.
24.
25.
Page no. 17
Anonymous. The Society of Cardiothoracic Surgeons of Great Britain
and Ireland. National Adult Cardiac Surgical Database Report. Final
draft. London: SCTS; 1998.
Roques F, Nashef SA, Michel P, Gauducheau E, de Vincentiis C,
Baudet E, et al. Risk factors and outcome in European cardiac surgery:
Analysis of the EuroSCORE multinational database of 19030 patients.
Eur J Cardiothorac Surg 1999;15:816‑22.
Nilsson J, Algotsson L, Höglund P, Lührs C, Brandt J. EuroSCORE
predicts intensive care unit stay and costs of open heart surgery. Ann
Thorac Surg 2004;78:1528‑34.
Toumpoulis IK, Anagnostopoulos CE, Swistel DG, DeRose JJ Jr.
Does EuroSCORE predict length of stay and specific postoperative
complications after cardiac surgery? Eur J Cardiothorac Surg
2005;27:128‑33.
Toumpoulis IK, Anagnostopoulos CE, DeRose JJ, Swistel DG.
Does EuroSCORE predict length of stay and specific postoperative
complications after coronary artery bypass grafting? Int J Cardiol
2005;105:19‑25.
Pinna Pintor P, Bobbio M, Colangelo S, Veglia F, Marras R, Diena M.
Can EuroSCORE predict direct costs of cardiac surgery? Eur J
Cardiothorac Surg 2003;23:595‑8.
Australian Society of Cardiac and Thoracic Surgeons Cardiac Surgery
Database Project‑Surgeon Report; 2004.
Collart F, Feier H, Kerbaul F, Mouly‑Bandini A, Riberi A,
Mesana TG, et al. Valvular surgery in octogenarians: Operative
risks factors, evaluation of Euroscore and long term results. Eur J
Cardiothorac Surg 2005;27:276‑80.
Guida P, Mastro F, Scrascia G, Whitlock R, Paparella D. Performance
of the European System for Cardiac Operative Risk Evaluation II:
A meta‑analysis of 22 studies involving 145,592 cardiac surgery
procedures. J Thorac Cardiovasc Surg 2014;148:3049‑57.e1.
Sadeghi MM, Arasteh M, Gharipour M, Nilfroush P, Shamsolketabi H,
Etesampour A, et al. Evaluation of accuracy of Euroscore risk model
in prediction of perioperative mortality after coronary bypass graft
surgery in Isfahan. J Res Med Sci 2011;16:787‑92.
Lovegrove J, Valencia O, Treasure T, Sherlaw‑Johnson C, Gallivan S.
Monitoring the results of cardiac surgery by variable life‑adjusted
display. Lancet 1997;350:1128‑30.
Yap CH, Reid C, Yii M, Rowland MA, Mohajeri M, Skillington PD, et al.
Validation of the EuroSCORE model in Australia. Eur J Cardiothorac
Surg 2006;29:441‑6.
Bhatti F, Grayson AD, Grotte G, Fabri BM, Au J, Jones M, et al.
The logistic EuroSCORE in cardiac surgery: How well does it predict
operative risk? Heart 2006;92:1817‑20.
D’Errigo P, Seccareccia F, Rosato S, Manno V, Badoni G, Fusco D, et al.
Comparison between an empirically derived model and the EuroSCORE
system in the evaluation of hospital performance: The example
of the Italian CABG Outcome Project. Eur J Cardiothorac Surg
2008;33:325‑33.
Kawachi Y, Nakashima A, Toshima Y, Arinaga K, Kawano H. Risk
stratification analysis of operative mortality in heart and thoracic aorta
surgery: Comparison between Parsonnet and EuroSCORE additive
model. Eur J Cardiothorac Surg 2001;20:961‑6.
Nashef SA, Roques F, Hammill BG, Peterson ED, Michel P,
Grover FL, et al. Validation of European System for Cardiac Operative
Risk Evaluation (EuroSCORE) in North American cardiac surgery.
Eur J Cardiothorac Surg 2002;22:101‑5.
Karunasumetta C, Laksanabunsong P, Slisatkorn W, Wongkornrat W,
Sakiyalak P, Thongcharoen P, et al. Validation of EuroSCORE for
coronary artery bypass grafting at Siriraj Hospital. J Med Assoc Thai
2012;95:1178‑83.
van Straten AH, Tan EM, Hamad MA, Martens EJ, van Zundert AA.
Evaluation of the EuroSCORE risk scoring model for patients
undergoing coronary artery bypass graft surgery: A word of caution.
Neth Heart J 2010;18:355‑9.