Modality-aware differentially private federated learning for partially observed multimodal healthcare prediction

Abstract

Multimodal healthcare prediction can benefit from combining structured electronic health records, medical imaging, and acquisition-related metadata, but these modalities are often incompletely observed across patients and unevenly distributed across clinical sites. Federated learning provides a mechanism for collaborative model training without centralizing patient records, yet privacy-preserving federated optimization remains challenging when clients differ in modality availability, update reliability, and data distribution. This study develops MoPA-FL, a modality-aware differentially private federated learning framework for partially observed multimodal healthcare prediction. The framework integrates modality-specific encoders, gated missing-modality fusion, branch-wise differentially private update perturbation, and server-side parameter-group aggregation guided by modality support and update reliability. MoPA-FL was evaluated through controlled federated simulations using the COVID Data for Shared Learning dataset, preserving 4479 structured-record admissions while leveraging an imaging-linked subset of 1859 patients. Under non-IID federated settings, MoPA-FL achieved an AUROC of 0.872 and an AUPRC of 0.683, with higher performance than representative federated, multimodal federated, and privacy-preserving baselines in the evaluated setting. At the configured privacy-budget setting of \(\epsilon =2\) , MoPA-FL achieved an AUROC of 0.847, compared with 0.801 for differentially private federated averaging. These budget values describe the operational accounting configuration; a privacy guarantee for the complete server-observed training transcript remains to be established. These findings suggest that privacy, modality incompleteness, and client heterogeneity should be addressed jointly in multimodal healthcare federated learning.