Lightweight infrared and visible image fusion via global selective state-space modeling

Abstract

Infrared and visible image fusion aims to effectively exploit the strength of the infrared modality in target saliency perception and the complementary capability of the visible modality in representing fine-grained texture details, which is crucial for complex scene understanding and downstream visual tasks. Existing deep learning methods achieve strong performance but often rely on complex cross-modal modeling, leading to large models and high computational costs, limiting real-time deployment. To address these challenges, this paper proposes a lightweight infrared and visible image fusion framework, termed GSSMFuse, based on Global Selective State-Space Modeling. In the feature extraction stage, depthwise separable convolutions are employed to capture local spatial structural information with low computational overhead, followed by a Mamba-based GSSM block to efficiently model long-range dependencies, enabling joint representation of local details and global semantic information. In the fusion stage, infrared and visible features processed by the GSSM block are directly combined and further integrated using lightweight convolution, effectively exploiting the complementary characteristics of the two modalities while maintaining high computational efficiency. Furthermore, a knowledge distillation-based training strategy, together with a structural consistency constraint, is introduced to enhance the fusion quality of the lightweight model by guiding the student network to inherit discriminative representations from the teacher model while preserving a compact architecture. Extensive experiments on the public M3FD dataset demonstrate that the proposed GSSMFuse consistently outperforms existing state-of-the-art fusion methods, while significantly reducing model parameters and computational complexity and achieving competitive performance in downstream object detection tasks.