DFANet: Dual-Branch Feature Fusion with Multi-Attention Decoding for Colonoscopy Polyp Segmentation

Abstract

Accurate colonoscopic polyp segmentation is essential for early colorectal cancer detection, yet remains challenging due to low contrast, specular highlights, blurred boundaries, and large variations in polyp appearance. We propose DFANet, a dual-branch feature aggregation framework that combines transformer-driven global semantic modeling MiT-B5 with CNN-based fine-grained representation learning EfficientNet-B5. A channel-spatial fusion module adaptively integrates multi-scale encoder outputs through element-wise aggregation, channel attention, and spatial attention, followed by a DSC Transformer that enhances contextual reasoning before decoding. The multi-stage decoder progressively restores resolution while refining structural details to produce accurate segmentation masks. Extensive experiments on five benchmark datasets demonstrate consistent performance gains, with DFANet achieving improvements of up to 3-7% in Dice and mIoU over state-of-the-art methods on Kvasir-SEG, CVC-ClinicDB, BKAI-IGH, ETIS-LaribPolypDB, and CVC-ColonDB.