A Deep-Learning-Embedded Sparse Variational Optimization Method for Hyperspectral and Multispectral Image Fusion

Abstract

Hyperspectral and multispectral image (HSI–MSI) fusion aims to reconstruct a high-spatial-resolution HSI by fusing an observed low-resolution HSI and a high-resolution MSI. While sparse variational optimization (VO)-based methods offer strong physical interpretability, their performance is often hindered by the limited representation capacity of hand-crafted priors in modeling complex spatial–spectral correlations. To address this, we propose a novel deep-learning-embedded sparse variational optimization (DLSpVO) framework. By formulating the fusion task within a spectral dictionary representation paradigm, we incorporate a dual-constraint mechanism for tensor coefficient estimation. Specifically, the tensor coefficient is parameterized by a deep neural network to introduce a deep implicit prior. Meanwhile, an explicit $\ell _{1}$-norm regularization term is imposed on the tensor coefficient to promote sparse representations that are consistent with the underlying linear spectral mixture model. To solve the resulting model, we develop a hybrid alternating direction method of multipliers-based algorithm. This optimization scheme seamlessly integrates a self-supervised zero-shot network initialization strategy to accelerate convergence, followed by iterative optimization of the deep network parameters and other variables. Extensive experiments on both simulated and real remote sensing datasets demonstrate that the proposed DLSpVO consistently outperforms state-of-the-art HSI–MSI fusion methods in preserving both spatial details and spectral fidelity. The source code will be made publicly available upon acceptance of the manuscript.