Diffusion model-based image restoration: interactive learning of determinism and stochasticity

Abstract

In recent years, diffusion models have emerged as a prominent research focus in the field of image restoration (IR) due to their demonstrated diversity and interpretability in generating images. IR aims to recover degraded images resulting from various conditions. However, the significant distributional discrepancies of different degraded images in the feature space present a challenge, making it difficult for existing diffusion models to simultaneously address the heterogeneity in restoration requirements across tasks. To overcome this limitation, we design a novel diffusion framework. This framework employs interactive modeling of determinism and stochasticity inherent in multiple image degradation processes. Consequently, the diffusion process effectively adapts to the specific demands of different tasks, whether prioritizing precision or diversity in the restored images. In contrast to previous deep learning-based IR methods, our approach provides stronger interpretability for the restoration process. Within this framework, the deterministic modeling component offers clear guidance for the reverse generative process of diffusion, ensuring restoration accuracy. Simultaneously, the stochastic modeling component enhances the model’s generalization capabilities under complex conditions, enabling it to handle diverse IR scenarios. We conduct experiments on four representative IR tasks: image deraining, image denoising, shadow removal, and low-light image enhancement. The proposed method achieves superior performance across all these tasks, demonstrating excellent generality and adaptability. Source code is available at https://github.com/AHU-psy/ILDS-mian.