FLMMIF: privacy-preserving federated multi-modal medical image fusion

Abstract

As a pivotal technique in smart healthcare, medical image fusion integrates complementary functional and structural information to facilitate accurate diagnosis and enhance clinical decision-making reliability. However, existing centralized methods typically raise serious data privacy concerns, while standard distributed approaches often fail to balance global generalization with local node personalization due to data heterogeneity. To address this, we propose FLMMIF, a privacy-preserving framework integrating a federated learning paradigm and low-rank adaptation for personalized and secure medical image fusion. During the local training phase, we utilize a dual-branch encoder and single-branch decoder, adopting a two-stage iterative strategy: initially training low-rank parameters to secure local personalization, followed by training full-rank parts to guarantee global baseline performance. Subsequently, this iterative process ensures that the model dynamically coordinates specific local features with general global knowledge before parameter transmission. Finally, we establish a metric-based aggregation mechanism on the server, FedIF, which evaluates the performance of uploaded models to assign higher aggregation weights to superior nodes for optimized global updating. Experimental results demonstrate that FLMMIF generates high-quality fusion results that effectively protect data privacy while achieving precise node-specific personalization.