SU-DVDNet: a video denoising method based on Swin Transformer UNet and FastDVDNet

Abstract

Addressing the limitations of traditional video denoising algorithms that only utilize spatial information from a single frame and ignore the temporal correlation between frames, as well as the existing multi-frame methods that require motion compensation, increasing computational overhead, and have a simple structure that struggles with high noise levels, this paper proposes an improved FastDVDNet algorithm based on SUNet, named SU-DVDNet. This algorithm takes five consecutive frames of noisy video as input, focuses on denoising the intermediate frames, and employs a two-stage network. In the first stage, the five frames are grouped into three-frame groups and input into a SUNet sub-network with shared weights for preliminary denoising. In the second stage, SUNet with independent weights is utilized to further optimize the results, fully exploiting spatiotemporal correlation without explicit motion compensation. Experiments were conducted on the Vimeo-90K dataset for training and testing, and the generalization ability was verified on the Davis2017 dataset. The noise levels were set at 15, 25, and 50, with MSE as the loss function. The results show that SU-DVDNet outperforms DRUNet, SUNet, and FastDVDNet in terms of PSNR and SSIM under different noise levels, balancing denoising and detail preservation. Ablation experiments verify that the two-stage structure significantly improves performance. This method provides an efficient technical solution for video denoising and enhancement.