UHFusion: Prompt-Guided Multi-Task Learning Network for Hyperspectral Image Fusion via Multi-Type Mixture-of-Experts

Abstract

Hyperspectral image (HSI) fusion aims to generate high-resolution HSI by integrating low-resolution hyperspectral data with auxiliary high-resolution sources (e.g., panchromatic (PAN), RGB, or MSI). While recent deep learning-based HSI fusion approaches have achieved promising results, they are typically designed for specific modality pairs and struggle to generalize across diverse fusion settings, leading to redundant architectures and limited adaptability. To overcome these limitations, we propose UHFusion, a prompt-guided multi-task learning (MTL) network for HSI fusion, which enables flexible multi-task adaptation within a single framework. Particularly, UHFusion introduces two key components: 1) a multi-type mixture-of-experts (MTMoEs) that decomposes HSI fusion into complementary spatial, spectral, and spatial–spectral reasoning processes, and dynamically composes expert responses conditioned on the auxiliary modality, allowing the model to selectively emphasize task-relevant spatial structures or spectral characteristics without architectural modification; and 2) a prompt-guided multi-task adaptation module (PMAM), which encodes modality-specific priors into learnable prompt embeddings and leverages a task relation graph to perform task-conditioned feature modulation, thereby improving the spatial–spectral fusion quality across diverse fusion tasks. Extensive experiments demonstrate that UHFusion achieves superior performance and generalization across multiple HSI fusion tasks and modalities, providing an effective all-in-one solution for HSI reconstruction. The code is available at: https://github.com/Jiahuiqu/UHFusion