Statistical modelling driven artificial intelligence for hospital discharge decision-making

Abstract

Effective discharge decision-making in healthcare is a critical yet complex process that significantly influences patient outcomes, hospital efficiency, and overall healthcare costs. Current methods often fail to capture the dynamic, evolving nature of patient care and exhibit inconsistencies in modelling approaches, particularly for length of stay (LOS) prediction and discharge decision-making. This thesis addresses these challenges through a systematic progression of research, culminating in the Unified Discharge Decision Model (UDDM) framework, which integrates novel predictive techniques and advanced statistical modelling. The research begins with an in-depth exploration of the Medical Information Mart for Intensive Care III (MIMIC-III) dataset, a publicly available critical care database developed by the MIT Lab for Computational Physiology in collaboration with the Beth Israel Deaconess Medical Center in Boston, Massachusetts. MIMIC-III contains comprehensive, de-identified health data associated with over 60,000 admissions between 2001 and 2012, covering demographics, vital signs, laboratory results, medications, procedures, and clinical notes. The richness and granularity of this dataset, along with its wide acceptance in academic research, make it a powerful foundation for building and evaluating advanced predictive models in hospital decision-making. Using this dataset, a composite super learner model was developed to predict patient LOS at the point of admission, significantly outperforming existing approaches with a root mean square error (RMSE) of 1.99 days, compared to the previously reported 9.5 days. This marked reduction in prediction error underscores the model’s potential to enhance discharge planning and resource utilisation, providing a robust baseline for subsequent phases of the research. Recognising inconsistencies in LOS classification intervals across prior studies, this thesis introduces a novel unsupervised discretisation method, leveraging exponential mixture models to segment LOS into interpretable intervals. By fitting the data to a mixture of exponential distributions using the Expectation-Maximisation algorithm, the method identifies cut points through intersections of the component distributions. This approach significantly improves classification accuracy and mutual information across machine learning models such as Random Forest, Neural Networks, and Gradient Boosting Machines. Additionally, it maintains a stability measure of 1, demonstrating its reliability and superiority over traditional methods such as Equal Width and Equal Frequency. To facilitate the adoption of this innovative discretisation approach, the disc.emm R package was developed. This tool encapsulates the methodology, addressing a critical gap in the R ecosystem by providing a dedicated solution for handling right-skewed and multimodal data distributions. The package enhances the efficiency and accuracy of data preprocessing pipelines, establishing itself as a valuable contribution to data science. Building on this foundation, the research introduces the Progressive Coxian Phase-Type Dynamic Markov (PCPDM) model, capturing the evolving nature of a patient’s hospital journey from admission to discharge or death. By structuring LOS into sequential phases, two PCPDM models were developed—one for discharge and one for mortality—integrated with a classifier to dynamically determine the appropriate pathway. This integration forms the Unified Discharge Decision Model (UDDM) framework, which demonstrated significant predictive performance improvements when evaluated on the MIMIC-IV dataset, achieving higher variance explanation (R-squared) and reduced prediction errors compared to baseline models. To support the implementation of the UDDM framework, the CoxianPhaseShiftR package was developed. This package addresses gaps in existing R tools by enabling the fitting of Coxian phase-type distributions and integrating dynamic Markov models. By optimising matrix exponential computations using efficient algorithms, it reduces computational complexity, ensuring seamless application in dynamic discharge decision modelling. Each stage of this research—from LOS prediction to discretisation, dynamic modelling, and tool development—was rigorously evaluated, consistently demonstrating superior performance and accuracy over existing methodologies. The findings contribute to advancing predictive healthcare analytics, optimising hospital operations, and improving patient care through data-driven decision-making tools. Thesis is embargoed until 31 July 2030.