For multi-modal image super-resolution (MISR), exploring cross-modal consistency is of vital importance. However, most existing consistency priors struggle to preserve high-frequency components and fail to provide generalizable regularization, often resulting in blurred edges or inaccurate textures. In this paper, we revisit the pivotal role of consistency prior in MISR task and present an important finding: the modality gap in the Laplacian response between guidance and target images does not conform to the widely adopted Gaussian or Laplacian distributions. Instead, it aligns better with T-distribution. Based on this insight, we propose a T-distribution formed Laplacian response Consistency (TLC) model. This model integrates a T-distribution based Multi-modal Consistency (TMC) prior with a learnable regularization term that operates between the guidance and target images. Additionally, we introduce a Multiplicative Degradation (MD) matrix to model the degradation process from high-resolution (HR) to low-resolution (LR) target images, thereby enabling adaptive non-uniform degradation modeling. The iterative optimization steps of the TLC model are subsequently unfolded into an interpretable network, termed TLCNet. The performance of TLCNet is evaluated on nine datasets across three MISR tasks, demonstrating its superior super-resolution performance compared to other state-of-the-art approaches. The visualization of intermediate features and the causal analysis of the guidance image further confirm its good interpretability.
Jingyi Xu, Xin Deng, Yutong Wang et al.· IEEE Transactions on Pattern...· 0 citations
The deep unfolding network represents a promising research avenue in fusion-based hyperspectral image super-resolution (HSI-SR). However, most current deep unfolding methodologies are anchored in idealized observation models, which overlook the degradation of the multispectral image (MSI), hindering their SR performance and practical applicability. To address this problem, this paper establishes a novel Dual-Degradation Fusion (D2-Fusion) model, which incorporates both HSI degradation and MSI blurring into the HSI-SR modeling process. Subsequently, we apply the second-order semismooth Newton algorithm to solve the optimization problem in D2-Fusion model. The solution steps are then mapped into an end-to-end trainable network, termed Blur-resistant Hyperspectral image Super-Resolution Network (BHSR-Net). To the best of our knowledge, the proposed network is the first successful attempt to consider MSI blurring artifacts in the HSI-SR task. It offers several distinct advantages: 1) The network structure maintains a strict mathematical correspondence with the optimization algorithm, ensuring each module retains strong physical interpretability; 2) The network exhibits superior SR performance and strong generalization ability on both standard and real-world scenarios across five datasets; 3) The network demonstrates excellent learning efficiency with a compact architecture, and its lightweight variant achieves comparable results with only 38K parameters. The code is available at https://github.com/Dou0405/BHSR-Net
Mai Xu, Yong Dou, Xin Deng et al.· IEEE Transactions on Image P...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
