Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Comparison of different statistical methods to predict Intensive Care Length of Stay Ilona Verburg Nicolette de Keizer Niels Peek Dept. Of Medical Informatics Academic Medical Center University of Amsterdam The Netherlands ESCTAIC 2012,Timisoara Background and objective Background Intensive Care Units (ICUs) assess their performance to improve quality and reduce costs Background Efficiency of care Effectiveness of care Case mix mortality length of stay 4-10-2012 2 Background and objective ICU Length of stay is influenced by case mix. Example: Length of stay (mean) 10 days 5 days Age (mean) 68 57 Medical vs surgical 80% medical 40% medical admission type (%) 20% surgical 60% surgical 4-10-2012 3 Background and objective Observed outcome Compare ICU Case mix Case mix 4-10-2012 Predictive model Expected outcome 4 Background and objective Background Models exist to predict ICU mortality (example APACHE IV) Few models exist to predict ICU Length of Stay (LoS) No consensus about best modelling method Objective Compare the performance of different statistical regression methods to predict ICU LoS. 4-10-2012 5 Data NICE registry Dutch National Intensive Care Evaluation (NICE) Registry of ICU admissions in the Netherlands (since 1996) All admissions from (voluntary) participating ICUs (>90%) Database Evaluating (systematically) the effectiveness and efficiency of ICUs in the Netherlands Identifying quality of care problems Quality assurance 4-10-2012 6 Data Data Patients admitted to ICUs participating NICE 2009 - 2011 84 ICUs Included patients Exclusion criteria APACHE IV exclusion criteria elective surgery 94,251 (42.4%) admissions 81,190 (86.1%) survivors 4-10-2012 13,061 (13.9%) non-survivors 7 Length of stay Distribution of Length of Stay in fractional days ICU survivors (n= 81,190) Median: 1.7 (days) Mean: 4.2 Standard deviation: 8.2 Maximum: 326.6 4-10-2012 ICU non-survivors (n= 13,061) Median: 2.4 (days) Mean: 5.9 Standard deviation: 10.2 Maximum: 139.0 8 ICU Length of Stay Distribution of discharge time 9 Modeling ICU length of stay Different methods to model ICU length of stay (in fractional days) Ordinary least square (OLS) regression LoS and Log-transformed LoS Most frequently used method in literature 4-10-2012 10 Modeling ICU length of stay Different methods to model ICU length of stay (in fractional days) Ordinary least square (OLS) regression LoS and Log-transformed LoS General linear models (GLM) Gaussian Gamma Poisson Negative binomial 4-10-2012 - difference with OLS is the log link function - LoS time until discharge - depending on chosen parameters positively skewed - LoS count data `-depending on chosen parameters positively skewed - property: expectation = variance → overdispersion - count data -depending on chosen parameters positively skewed - generalisation of poisson 11 Modeling ICU length of stay Different methods to model ICU length of stay (in fractional days) Ordinary least square (OLS) regression LoS and Log-transformed LoS General linear models (GLM) 4 different families Gaussian Gamma Poisson negative binomial Cox proportional Hazard (Cox PH) regression No assumptions on the shape of the distribution Omits the need of transform the outcome 4-10-2012 12 Modeling ICU length of stay Selection of covariates Starting with large set of variables Known relationship with LoS (literature) Stepwise backwards elimination of variables Included case mix Demographics Age Gender Admission type Diagnoses (APACHE IV) Severity of illness (APACHE IV severity-of-illness score) Different comorbidities (21) 4-10-2012 13 Validation Good prediction Performance measures Cov(Y , Yˆ ) 2 Squared Pearson correlation = R = ˆ (Y ) (Y ) Root Mean squared prediction error (RMSPE) = Relative BIAS = 1 1 Eyk y k n k n k 1 yk n k Relative mean absolute prediction error (MAPE) = 4-10-2012 2 High ↑ 1 2 Ey y k k n k Low ↓ Low ↓ - or + 1 Eyk y k n k 1 yk n k Low ↓ 14 Validation Validation Performance measures calculated on original data Correcting for optimistic bias 100 bootstrap samples 4-10-2012 15 Results coefficients Covariates survivors chronic dialysis cva diabetes resperatory insufficient spline Aps (1) spline Aps (2) spline Aps (3) Covariates non-survivors chronic dialysis cva diabetes resperatory insufficient spline Aps (1) spline Aps (2) spline Aps (3) 4-10-2012 OLS reg OLS reg GLM: GLM: GLM: negative GLM: Cox los log los gaussian poisson binomial Gamma PH -1.04 -0.16 -0.25 -0.26 -0.28 -0.28 0.31 0.74 0.1 0.13 0.18 0.26 0.26 -0.3 -0.34 -0.01 -0.07 -0.06 -0.04 -0.04 0.03 0.38 0.03 0.06 0.09 0.15 0.15 -0.11 5.55 0.64 1.74 1.65 1.61 1.61 -1.52 11.07 1.09 3.16 2.78 2.64 2.64 -2.57 15.98 0.99 2.07 2 2.08 2.08 -1.79 OLS reg los -0.68 0.35 -0.51 -5.59 -6.08 -6.47 OLS reg log los GLM: GLM: GLM: negative GLM: Cox gaussian poisson binomial Gamma PH 0.15 0.08 -0.18 -0.15 -0.12 -0.12 0.09 0.03 0.05 0.05 0.06 0.06 -0.05 -0.03 -0.11 -0.1 -0.09 -0.09 0.07 -0.43 -0.94 -0.84 -0.8 -0.8 0.7 -0.73 -1.09 -1.26 -1.53 -1.55 1.54 -0.84 -1.64 -1.76 -1.87 -1.88 1.83 16 Results validation ICU survivors OLS regression (LoS) R2 0.174 OLS regression (log(LoS)) 0.183 7.714 -0.400 0.674 GLM Gaussian 0.197 7.335 0.001 0.771 GLM Poisson 0.194 7.349 0.000 0.769 GLM Negative Binomial 0.186 7.388 0.005 0.773 GLM Gamma 0.184 7.407 0.005 0.773 Cox PH regression 0.097 9.002 -0.693 0.938 RMSPE Relative BIAS Relative MAPE 7.448 0.008 0.812 Mean observed > mean expected Underestimation of mean LoS 4-10-2012 17 Results validation ICU non-survivors OLS regression (LoS ) R2 0.107 RMSPE 9.618 Relative BIAS 0.005 Relative MAPE 0.891 OLS regression (log(LoS)) 0.107 10.213 -0.510 0.762 GLM Gaussian GLM Poisson GLM Negative Binomial GLM Gamma Cox PH regression 0.134 9.462 -0.009 0.868 0.128 0.12 0.112 0.075 9.504 9.545 9.602 11.388 0.000 -0.001 -0.001 -0.808 0.872 0.872 0.877 0.906 4-10-2012 18 Conclusion and discussion Difficult to predict ICU LoS Influenced by admission and discharge policy Seasonal pattern for admission and discharge time Skewed to the right GLM models shows best performance Poorest performance found for Cox PH regression Large relative bias was found for OLS regression of log-transformed LoS Differences in performance between models not statistically tested 4-10-2012 19 Conclusion and discussion Similar study for CABG patients (Austin et al.), with comparable results Different patient type Different distribution of length of stay Future research Different models for survivors and non-survivors combining with mortality in one prediction Statistical methods to predict ICU LoS developing a model for benchmarking purposes 4-10-2012 20 Thank you for your attention! Questions? 4-10-2012 21 APACHE IV Exclusiecriteria • Age < 16 • ICU admission < 4 hours • Hospital admission >365 days • Died during admission • Readmissions • Admissions from CCU/IC other hospital • No diagnose • Burns • Transplantations • Missing hospital discharge 4-10-2012 22