FedSCF: Sensitivity-aware Collaborative Fusion for Personalized Federated Learning in Medical Image Classification

Abstract

In smart healthcare scenarios, patient data are distributed across different medical institutions. Due to privacy protection and data security constraints, such data cannot be directly shared, which consequently affects the training performance of medical image models. Federated Learning (FL) enables collaborative model training among multiple parties without exchanging raw data, allowing models to leverage multi-source data to improve overall performance and generalization capability. However, under non-independent and identically distributed (non-IID) conditions, significant discrepancies among client data can lead to unstable convergence and performance degradation during training. Personalized Federated Learning (PFL) alleviates this issue to some extent. Nevertheless, existing methods mostly rely on coarsegrained parameter partitioning, which fails to capture internal parameter heterogeneity and lacks effective modeling of inter-client similarity, thereby limiting the effectiveness of collaboration. We propose a personalized federated learning framework, FedSCF, which models client heterogeneity at the parameter level. Specifically, a relative perturbation-based sensitivity evaluation is designed to identify critical parameters for personalized modeling, while the remaining parameters participate in cross-client sharing. The overlap of critical parameter masks is utilized to characterize inter-client similarity, enabling the construction of a dynamic collaboration structure. An adaptive threshold is further introduced to select collaborative neighbors, allowing clients with similar data distributions to form stable collaboration relationships. During the aggregation stage, a dual-path strategy is developed to jointly learn globally shared knowledge and personalized features, and a mask-guided parameter-wise fusion mechanism is employed to complete model updates. Experimental results on the Kvasir and Dermoscopic datasets demonstrate that FedSCF outperforms current state-of-the-art methods under heterogeneous data settings.