MDCF-Net: Multiscale Dual-Domain Collaborative-Aware Feature Fusion Network for Hyperspectral and Multispectral Image Fusion

Abstract

Deep learning-based image fusion methods have achieved remarkable progress in the fusion task of low-resolution hyperspectral images (LR-HSI) and high-resolution multispectral images (HR-MSI). However, existing solutions still suffer from limitations, particularly in multiscale feature representation and cross-scale information fusion. To resolve these limitations, we propose a multiscale dual-domain collaborative-aware feature fusion network (MDCF-Net) for LR-HSI and HR-MSI fusion. To overcome the insufficiency of multiscale feature representation, MDCF-Net extracts features by constructing a multiscale patch-based dual-domain feature extraction (MPDFE) module. Specifically, the MPDFE module employs dual-domain feature extraction (DDFE) modules with varying receptive fields to capture multiscale spatial and frequency information from source images. The DDFE module consists of a spatial feature extraction module and a frequency feature extraction module, which jointly exploit fine-grained spatial information and global frequency distribution characteristics through deep convolution and Fourier transform. By leveraging the complementarity between spatial and frequency domains, the model enhances the completeness and discriminability of fused features, thereby providing more expressive intermediate representations for subsequent cross-scale feature fusion. Furthermore, to resolve the issue of insufficient cross-scale information integration, we build a dual-domain collaborative perception fusion module. This module enables adaptive weighted combination of spatial and frequency-domain features at multiple scales, significantly improving the model’s joint representation ability. Evaluations on three simulated datasets and one real dataset show that our approach outperforms current state-of-the-art methods in both quantitative scores and subjective visual quality under various scenarios.