GenMSLnet: Generative Modality-Specific Learning-Enhanced Collaborative Fusion Network for Hyperspectral and SAR Joint Classification

Abstract

Hyperspectral (HS) images and synthetic aperture radar (SAR) data provide complementary information for land cover interpretation. However, their cross-modal collaboration remains insufficient in multisatellite observation systems. Moreover, both remote sensing (RS) modalities are sensitive to environmental interference, hindering critical feature acquisition. To overcome these challenges, a generative modality-specific learning-enhanced collaborative fusion network called GenMSLnet is proposed for HS and SAR joint classification. Our core innovation is modality-specific feature learning, directly guided by the RS inherent properties. The framework couples a multiscale dual-stream generative adversarial network (MDGAN) for modality-specific feature learning with a transformer-based dual-path cross-attention (TDCA) network for complementary fusion. MDGAN comprises dual streams for HS image (HSI) and SAR data. The former captures spectral–spatial details by multiscale feature extraction and attention, while the latter employs orientation-aware sampling to capture geometry- and orientation-sensitive backscattering signals. A residual separable U-Net discriminator (RSU-NetD) incorporating depthwise separable channel attention residual blocks (DSCA ResBlocks) is considered to strengthen feature discrimination. TDCA balances RS modality contributions by introducing a novel gated bidirectional dynamic cross-attention mechanism. Experiments demonstrate that GenMSLnet outperforms state-of-the-art techniques quantitatively and qualitatively. Codes will be available at https://github.com/zhanghongzhan-123/GenMSLnet