P4VC: Positive Perturbation based Perceptual Preprocessing Framework for Video Compression.

Abstract

Recent advancements in deep learning have significantly propelled the enhancement of video compression frameworks, encompassing both encoder-side and postprocessing methods. However, these extensively explored methodologies have reached their limits, offering diminishing returns for further improvement. To overcome the constraints of the above optimization patterns, we propose to step beyond conventional frameworks and focus on preprocessing ahead of compression. For preprocessing, the black-box nature of video codecs introduces challenges for deep learning-based optimization: 1) the absence of ground-truth preprocessed videos, and 2) the lack of end-to-end optimization mechanism. To address these challenges, we introduce P4VC, a Positive Perturbation based Perceptual Preprocessing framework that generates adaptive perturbations before compression to enhance the rate-perception trade-off. Specifically, P4VC develops an alternative updating optimization scheme with 1) individual optimization phase that employs an attack-based method to generate multiple codec-friendly positive perturbations for each training sample, directly targeting the practical codec, and 2) universal optimization phase that trains a lightweight preprocessing network to generalize across arbitrary videos, guided by the positive perturbations and developed distribution-aware adversarial learning scheme. Extensive experiments across five codecs, two datasets and six perceptual metrics, demonstrate that P4VC consistently achieves significant compression gains, superior generalization, and real-time preprocessing at 241 FPS.