LightVM-SparseUNet: a lightweight medical image segmentation framework via Vision Mamba and sparse attention

Abstract

To address the challenges of high parameter redundancy and prohibitive computational complexity inherent in traditional convolutional neural networks and Transformer architectures—which impede deployment on resource-constrained edge medical devices—this paper proposes LightVM-SparseUNet, an ultra-lightweight medical image segmentation framework based on state space models. The core innovations are twofold: First, a Multi-path Visual Mamba module is designed to significantly enhance feature extraction efficiency via a linear-complexity inference mechanism while maintaining feature channel integrity. Second, a sparse-sampling self-attention mechanism is integrated into the U-shaped skip connections, enabling the precise capture of long-range spatial dependencies and mitigating spatial information loss at minimal computational cost. Experimental results demonstrate that LightVM-SparseUNet achieves segmentation competitive with state-of-the-art large-scale models across two authoritative public datasets. Critically, the proposed model achieves extreme lightweights, with a parameter count of only 0.08 M and a computational overhead of merely 0.16 GFLOPs.Our method is highly practical, and the code can be found at https://github.com/yjzbkl/LightVM-SparseUNet.