Mortality prediction in intensive care units including premorbid functional status improved performance and internal validity.

OBJECTIVE

Prognostic models are key for benchmarking intensive care units (ICUs). They require up-to-date predictors and should report transportability properties for reliable predictions. We developed and validated an in-hospital mortality risk prediction model to facilitate benchmarking, quality assurance, and health economics evaluation.

STUDY DESIGN AND SETTING

We retrieved data from the database of an international (Finland, Estonia, Switzerland) multicenter ICU cohort study from 2015-2017. We used a hierarchical logistic regression model that included age, a modified Simplified Acute Physiology Score-II, admission type, premorbid functional status, and diagnosis as grouping variable. We used pooled and meta-analytic cross-validation approaches to assess temporal and geographical transportability.

RESULTS

We included 61,224 patients treated in the ICU (hospital mortality 10.6%). The developed prediction model had an area under the receiver operating characteristic curve 0.886, 95% confidence interval (CI) 0.882-0.890; a calibration slope 1.01, 95% CI (0.99-1.03); a mean calibration -0.004, 95% CI (-0.035-0.027). While the model showed very good internal validity and geographic discrimination transportability, we found substantial heterogeneity of performance measures between ICUs (I-squared: 53.4% to 84.7%).

CONCLUSION

A novel framework evaluating the performance of our prediction model provided key information to judge the validity of our model and its adaptation for future use. WHAT IS NEW?: : Key findings; Our mortality prediction model-which combined established clinically relevant predictors with premorbid functional status and diagnoses as modeling variables-showed very good internal validity, geographic discrimination and temporal transportability, with a substantial heterogeneity of performance measures between ICUs. What does this add to what is known? Premorbid functional status and diagnosis are known predictors of ICU-relevant study outcomes, but are not regularly implemented in established scoring systems. The inclusion of this information showed increased predictive model performance compared to predictions from established risk scoring systems, while showing good internal validation and transportability properties. What is the implication and what should change now? To the best of our knowledge, this is one of the first development and validation studies to investigate geographical and temporal transportability properties of an ICU mortality prediction model. Transportability properties are key in the reliable monitoring and benchmarking of ICUs and for their planning. They provide an important piece of information about the model validity in other study populations and settings, and should be quantified in future validation studies of ICU prediction models.