TSPFusion: Tri-stream and prototype network for learning detail-semantic fusion in medical image segmentation
TL;DR
A tri-stream interaction paradigm replacing symmetric skip connections with directional fusion among semantic, spatial, and decoder-propagated streams at each decoding stage, and a Global Prototype Bank that captures dataset-level anatomical regularities via attention-based retrieval and gated EMA updates, providing persistent semantic priors across images.
Abstract
Multi-scale feature fusion is a cornerstone of encoder-decoder architectures in medical image segmentation, yet effectively integrating representations across stages remains a significant challenge due to the inherent semantic–spatial gap. Deep features encode abstract semantic context but lack spatial precision, whereas early-stage features preserve fine-grained details but suffer from limited semantic discriminability. Existing fusion mechanisms, which often rely on symmetric aggregation or simple skip connections, fail to explicitly model the semantic-to-spatial guidance necessary for precise alignment. To address this, we propose a Tri-stream Prototype Fusion Network (TSPFusion) that introduces three key innovations: (i) a tri-stream interaction paradigm replacing symmetric skip connections with directional fusion among semantic, spatial, and decoder-propagated streams at each decoding stage; (ii) a Global Prototype Bank (GPB) that captures dataset-level anatomical regularities via attention-based retrieval and gated EMA updates, providing persistent semantic priors across images; and (iii) a Detail–Semantic Feature Aligner (DSFA) that performs semantic-guided refinement of spatial features prior to fusion, preventing feature interference from direct concatenation. Additionally, an Adaptive Pyramid Context Decoder module aggregates multi-scale information with resolution-aware dynamic pooling, and a Gradient-Gated Spatial Attention head enforces boundary-sensitive structural consistency. Extensive experiments on four medical imaging benchmarks (CT and Ultrasound) demonstrate that TSPFusion achieves state-of-the-art performance 97.82±0.85% DSC on COVID19 lung CT, 81.76% DSC on COVID19-Seg, and 88.16% mDice on cross-dataset BUSI→\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\rightarrow $$\end{document}STU, while maintaining a compact 5.92M parameter footprint.