ZoomDiff is proposed, a high-fidelity diffusion model that leverages dual-camera inputs in both latent and pixel spaces for photo-realistic transitions that outperforms state-of-the-art methods quantitatively and qualitatively.
Abstract
Digital zoom transitions between dual cameras often exhibit conspicuous discontinuities in geometric structure and chromatic consistency, degrading the user experience. While recent dual-camera smooth zoom (DCSZ) methods attempt to mitigate this by fine-tuning frame interpolation (FI) models on DCSZ data, they struggle with the large cross-view disparities and complex geometric transformations. Considering that the generative prior of diffusion models is suitable for addressing this problem, we explore their application to DCSZ. However, naively applying existing diffusion-based FI models still yields low-fidelity transitions due to insufficient conditional guidance, high-frequency information loss during VAE encoding, as well as inadequate temporal consistency. To address this, we propose ZoomDiff, a high-fidelity diffusion model that leverages dual-camera inputs in both latent and pixel spaces for photo-realistic transitions. Specifically, we first strengthen dual-image conditional guidance during the multi-step denoising process to improve geometric consistency. Then we inject flow-aligned multi-scale features from the VAE encoder into the VAE decoder to recover high-frequency details, where flow-guided temporal consistency supervision are introduced to produce more smooth transitions. Extensive experiments on both synthetic and real-world datasets demonstrate that ZoomDiff outperforms state-of-the-art methods quantitatively and qualitatively. Project page: https://jiayi-hit.github.io/ZoomDiff.github.io/.
This work proposes a novel 3D-aware video restoration framework designed to enhance the quality of sparse 3DGS reconstruction and introduces a camera-conditioned geometric prior that guides the network toward geometrically grounded restoration that remains coherent across viewpoints.
Xinhui Liu, Can Wang, Wei Jiang et al.· 1 citation
This work introduces an exact analytical pixel-to-texel mapping that aligns diffusion trajectories across multiple viewpoints, and utilizes High-Resolution Latent Textures as a persistent canvas for gradually denoised textures, while camera views perform the denoising steps in latent pixel space.
Athanasios Tragakis, M. Aversa, D. Ivanova et al.· 0 citations
This work proposes Prior-driven Patch-based Bilateral Compensation (P2BiC), a memory-efficient UHD restoration framework that processes UHD images as patches via an in-network feature-space cropping computation path while maintaining global coherence.
Ze-Yu Xiao, Yang Zhao, Yu Liu et al.· International Journal of Com...· 0 citations
This paper proposes a novel iterative refinement framework based on a video diffusion model to improve the completeness and consistency of dynamic 4D scenes, and substantially outperforms existing baselines.
UGDiff, a novel diffusion guidance paradigm designed to further improve the perception-distortion balance, is introduced, which first estimates the reconstruction uncertainty of the latent features corresponding to a high-fidelity image and guides the diffusion process to selectively restore high-frequency details in h...
Ren Wang, Yung-Yu Chuang· International Conference on...· 0 citations
This work introduces additional 3D spatial prior information into both the training and inference stages of video diffusion models to enhance the spatial structural consistency of generated videos and introduces an energy-function guidance strategy (Warp-Guidance) driven by warping priors during denoising.
Hong-Zhou Zhu, Xue Yang, Min Zhao et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.