Multimodal Deep Attention Fusion of Longitudinal EHR and Diagnostic Imaging for Real-Time Sepsis and Deterioration Prediction in Critical Care
TL;DR
This work presents MedCross-Fusion, a novel multimodal deep learning framework that dynamically integrates irregular, longitudinal EHR time-series data with diagnostic chest radiography with robust, real-time risk stratification capable of reducing diagnostic latency in critical care settings.
Abstract
Early detection of clinical deterioration and septic shock in Intensive Care Units (ICUs) remains one of the most pressing challenges in modern critical care medicine. Sepsis is characterized by a dysregulated host response to infection, leading to life-threatening organ dysfunction. While traditional early warning scoring systems (e.g., SOFA, MEWS, APACHE-IV) rely on static, threshold-based heuristics of isolated physiological vital signs, they suffer from high false-alarm rates and lack the ability to assimilate continuous, high-frequency temporal trajectories alongside high-dimensional diagnostic imaging. In this work, we present MedCross-Fusion, a novel multimodal deep learning framework that dynamically integrates irregular, longitudinal electronic health record (EHR) time-series data with diagnostic chest radiography (CXR). Our architecture comprises a Temporal Cross-Attention Transformer coupled with a lightweight Swin-Transformer visual backbone, linked through a cross-modal gating mechanism that dynamically weights visual pathology representations against continuous physiological fluctuations. We train and evaluate MedCross-Fusion on two extensive clinical datasets: MIMIC-IV (comprising over 45,000 ICU admissions) and the multi-center eICU Collaborative Research Database (over 200,000 ICU admissions). MedCross-Fusion achieves an Area Under the Receiver Operating Characteristic curve (AUROC) of 0.932 (95% CI: 0.925–0.939) and an Area Under the Precision-Recall Curve (AUPRC) of 0.684 for predicting sepsis onset 6 hours prior to clinical diagnosis, significantly outperforming unimodal recurrent baselines (AUROC 0.841) and state-of-the-art multimodal fusion architectures (AUROC 0.887). Furthermore, we integrate Integrated Gradients and Token-wise Attention Rollout to provide interpretable clinical explanations. Our results demonstrate that MedCross-Fusion delivers robust, real-time risk stratification capable of reducing diagnostic latency in critical care settings.