Decentralized multimodal medical image segmentation via federated learning and segment anything model

Abstract

Medical image segmentation faces critical challenges, including poor multi-center data generalization, limited multimodal handling, and high privacy risks. This study proposes a novel decentralized multimodal medical image segmentation method (FL-SAMMed) that integrates federated learning with the SAM-Med2D model. Firstly, the pre-trained SAM-Med2D model, optimized for feature learning across heterogeneous medical images, was leveraged to construct a multimodal segmentation model. A custom federated learning framework was then designed to enable privacy-preserving distributed training on multi-center data. Experimental results on multiple medical image datasets confirm that the proposed FL+SAM-Med2D method outperforms baseline approaches, achieving an average Dice score of 0.862 ± 0.028 and a mean HD95 of 6.68 ± 1.20 mm. These results highlight substantial improvements in segmentation accuracy and robustness, especially for cardiac MRI segmentation scenarios. Furthermore, the method mitigates data heterogeneity and ensures strict privacy compliance.