Hyperspectral and Multispectral Image Fusion via Deep Generalised Linear Mixed Model
TL;DR
This work proposes a novel fusion framework based on the generalised linear mixed model theory for hyperspectral–multispectral image fusion that improves performance compared to state‐of‐the‐art methods but also provides improved interpretability and computational efficiency.
Abstract
Hyperspectral–multispectral image fusion is an incomplete information fusion problem, and directly embedding information can lead to modality‐specific responses and cause spectral distortion, whereas the strong interband redundancy of hyperspectral images makes full‐dimensional fusion inefficient. To address the aforementioned issues, we propose a novel fusion framework based on the generalised linear mixed model theory. Specifically, high‐dimensional hyperspectral images can be composed of a low‐dimensional abundance matrix, which describes the pixel‐wise spatial proportions of the spectral components, and an endmember matrix, which contains their spectral signatures, and information fusion is achieved by correcting the spatial abundance representation with HR‐MSI guidance and refining the spectral endmember representation. In the proposed method, we generate an initial abundance matrix from the interpolated LR‐HSI and estimate the corresponding endmember matrix using regularised least squares. Considering that abundance and endmember matrices directly calculated from low spatial resolution hyperspectral images are unreliable, we propose an abundance matrix correction mechanism and endmember matrix fusion strategy. Repeating the solving and refinement stages progressively reduces the initialisation error and yields coarse‐to‐fine fusion. The proposed method not only improves performance compared to state‐of‐the‐art methods but also provides improved interpretability and computational efficiency. The comparison on multiple datasets also demonstrates the effectiveness of the proposed method across multiple datasets. The source code will be accessed at: https://github.com/jiaming‐wang/Project .