Delving Deep Into Unraveling Spatio-Temporal Forgery Cues for Face Forgery Video Detection.

Abstract

Face Forgery Video Detection is a critical yet challenging task in determining whether a digital facial video is authentic or forged. Existing methods often rely on isolated spatial or coarsely fused spatio-temporal features, which are insufficient to mine spatio-temporal forgery cues and lead to unsatisfactory generalization performance. To this end, we strive to unravel general spatio-temporal forgery cues across three progressive levels: (1) spatial regional suspicions, (2) temporal momentary anomalies and gradual inconsistencies, and (3) spatio-temporal cumulative distortions. Accordingly, we design a hierarchical cooperation module to identify regional suspicion cues by coordinating intra-frame features across different Transformer layers of Image Encoder. Then, we devise a gradual future guidance module to unravel temporal cues by iteratively aggregating historical anomaly cues and gradually propagating inconsistency cues into future frames. Finally, we introduce a historical review module that unravels distortion cues via momentum accumulation and spatio-temporal review from future to historical frames. These three modules form Spatio-Temporal Forgery Cue Unraveling (STFCU) framework, sequentially highlighting spatial discriminative features by unraveling spatio-temporal forgery cues bidirectionally between historical and future frames. Extensive experiments and ablation studies demonstrate effectiveness of STFCU method, achieving state-of-the-art performance across diverse unseen datasets and manipulation methods.