Continuous prediction of prolonged intensive care unit stay from longitudinal electronic health record data: development and external validation
Abstract
Abstract Objectives To develop and externally validate Prognostic Estimator of RiSk for prolonged ICU STay (PERSIST), a model that continuously updates the probability of an intensive care unit (ICU) stay longer than 7 days during the first week of ICU admission. Materials and Methods We developed PERSIST, an XGBoost model, on 49 255 adult ICU stays from the Mount Sinai Health System (MSHS; 2021-2025) and externally validated it in 72 156 Medical Information Mart for Intensive Care–IV (MIMIC-IV) stays (2008-2019). Sixty routinely available electronic health record (EHR) features were aggregated hourly, with predictions every 4 hours from preceding data, benchmarked against L2-regularized logistic regression. We assessed discrimination, calibration, net benefit, SHapley Additive exPlanations attributions, and subgroup performance. Results Prolonged stay occurred in 16.1% of MSHS and 11.1% of MIMIC-IV admissions. PERSIST achieved macroaveraged areas under the receiver operating characteristic curve of 0.80 (95% confidence interval [CI], 0.79-0.81) and 0.78 (0.77-0.78), respectively, compared with 0.75 (0.73-0.76) and 0.74 (0.73-0.74) for logistic regression. Corresponding areas under the precision-recall curve were 0.73 and 0.68. Brier scores were 0.14 and 0.13. Across modeled thresholds, PERSIST provided greater net benefit than treat-all and treat-none strategies. Discrimination was similar across reported age, sex, and racial groups. Discussion Prognostic Estimator of RiSk for prolonged ICU STay transported across 2 health systems differing in era, case mix, and outcome prevalence, and outperformed logistic regression under identical conditions. Net benefit was greatest during the early ICU days, when few patients have declared a prolonged trajectory and a risk estimate can still inform planning. Conclusion Continuously updated, EHR-based prediction of prolonged ICU stay is feasible and generalizable. Prospective evaluation should determine whether it improves ICU care coordination and capacity planning.