UniHyper: A Unified Multimodal Classification Framework for Hyperspectral Images Under Diff-Labeled Scenes

Abstract

Land-cover classification using multimodal remote sensing imagery is a fundamental task in Earth observation. Yet, current methods face two major challenges: the tension between modality consistency and specificity preservation in feature fusion, and the lack of a universal framework adaptable to diverse labeled scenarios (e.g., fully supervised, semi-supervised, noisy-label, weakly supervised, and unsupervised settings). To address these issues, this article proposes UniHyper, a unified multimodal classification framework that explicitly separates representation learning from the subsequent classification process. For representation learning, we introduce a Fourier-orthogonal feature decoupling strategy (FoFD) strategy to separate modality-invariant and modality-unique features, mitigating the specificity erosion problem in contrastive learning. Furthermore, a Fourier-orthogonal channel attention network (FoCA-Net) is designed to enhance cross-modal consistency while preserving discriminative characteristics via frequency-domain attention. For classification, a flexible teacher–student classifier is developed to process the different supervised tasks, while a dynamic prototype clustering strategy is proposed for the unsupervised task. Extensive experiments on multimodal remote sensing datasets demonstrate that UniHyper achieves state-of-the-art performance across multiple labeled scenarios, offering a generalized solution for robust land-cover classification.