Aug 2026· International Conference on Multimedia Analysis and Pattern Recognition· pp. 622-627· 0 citations· 40 references
TL;DR
TemporalTalk is presented, which closes this gap with three training-time mechanisms: a face-masked temporal consistency loss with stop-gradient and delayed activation, a multi-head self-attention module that replaces the fixed convolutional attention over the audio context window, and cosine learning-rate annealing with linear warmup.
Abstract
NeRF-based talking head methods can render individual frames with impressive fidelity, yet the assembled videos often flicker. The reason is structural: each frame is optimized independently, so nothing in the training objective ties frame t to frame t − 1. We present TemporalTalk, which closes this gap with three training-time mechanisms: a face-masked temporal consistency loss with stop-gradient and delayed activation, a multi-head self-attention module that replaces the fixed convolutional attention over the audio context window, and cosine learning-rate annealing with linear warmup. None of the three touches the inference path, so runtime cost is unchanged. On the May benchmark, TemporalTalk reduces temporal flicker measured by tLPIPS (the mean Learned Perceptual Image Patch Similarity between consecutive generated frames) by 24% relative to SyncTalk, while also improving PSNR by 0.14 dB, LPIPS (Learned Perceptual Image Patch Similarity) by 3.0%, and landmark distance by 2.3%. An ablation study shows that all three mechanisms contribute, with the temporal loss accounting for the largest share of the gain.
Local audio-visual deepfakes modify only a short word or phrase within an otherwise authentic video. Because the manipulated evidence occupies a small temporal region, global aggregation weakens detection cues, while accurate start/end estimation becomes as important as video-level classification. We present LocTFL, a...
Hua-Ye Wang, Bo Wu, Long-Teng Jiang et al.· Proceedings of the 2nd Deepf...· 0 citations
TT-VidT leads Jester, Something-Something V2, ARID, and Diving48 fine-tuning simultaneously, improving over the strongest non-TT row by 54% ~ 121%, while using 48% fewer encoder FLOPs than DisMo and 55% fewer than VideoMAE or V-JEPA2.
Shih-Ying Yeh, Daniel Z. Kaplan, Xue-Hai Wang et al.· arXiv.org· 0 citations
Co-speech gesture video generation aims to synthesize realistic, audio-driven portrait videos from a single reference image and a speech signal. Despite advances in visual quality, diffusion-based approaches still suffer from degraded local details and imperfect audio-visual alignment. We present PAGE (Parts-Aware Guid...
Ya-Biao Wang, Jia-Fu Wu, Y. Xu et al.· IEEE transactions on circuit...· 0 citations
About 63 million people in India live with significant hearing impairment, yet Indian Sign Language (ISL) is still poorly served by assistive digital tools even though it is central to communication within the Deaf and Hard-of-Hearing (DHH) community. We present STANet, a compact hybrid model for isolated ISL recogniti...
Surbhi Maheshwari, Amit Sinhal, Arpana Sinhal· Discover Computing· 0 citations
Large-scale vision-language pretrained models like CLIP, paired with parameter-efficient fine-tuning techniques, have emerged as promising solutions for image-to-video transfer in video action recognition. However, existing methods often prioritize strong supervised performance at the expense of transferability and gen...
Meng-Meng Wang, Ze-Yi Huang, Bo-Yuan Jiang et al.· IEEE Transactions on Pattern...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.