The method sets a new state of the art in 4D reconstruction and seamlessly generalizes to in-the-wild videos with large viewpoint changes and dynamic motions, improving both novel-view synthesis and the underlying 3D motion.
Abstract
We present World from Motion, a method for generating freely renderable dynamic 3D Gaussian representations from monocular videos. Our approach conditions a video model on dense, pixel-aligned renderings that encode appearance, geometry, and 3D scene motion along both input and target camera trajectories to correct rendering artifacts and fill in missing regions from an initial reconstruction. To train this model, we construct a dataset of aligned multiview video pairs and dynamic 3DGS representations, with simulated artifacts characteristic of monocular reconstruction. At test time, we distill the model's generations, including newly observed regions and motions, back into a single consistent, high-quality dynamic 3DGS, improving both novel-view synthesis and the underlying 3D motion. Our method sets a new state of the art in 4D reconstruction and seamlessly generalizes to in-the-wild videos with large viewpoint changes and dynamic motions.
This paper proposes a semantics-guided scene decoupling module that separates Gaussian primitives into static and dynamic components based on motion vectors, and introduces a motion-aware densification module for motion compensation, which alleviates the incomplete rendering of dynamic objects caused by insufficient sp...
Chulin Zhao, Xue Wang, Guo-Qing Zhou et al.· IEEE Transactions on Visuali...· 0 citations
AniGS is presented, a method for scene-level animation of 3D Gaussian Splatting (3DGS) reconstructions that adds subtle, distributed dynamics, e.g., vegetation motion, while preserving rigid structures in reconstructed environments.
OmniX achieves state-of-the-art performance on dense 3D point trajectory prediction and 3D point tracking, while also demonstrating competitive results on video depth estimation and camera pose estimation.
Yanqin Jiang, Tengfei Wang, Zhengwei Wang et al.· arXiv.org· 3 citations· ⚡1
This work proposes a framework known as Video-Generation Environment Representation (VGER), which leverages the advances of large-scale video generation models to generate a moving camera video conditioned on the input image, and demonstrates its ability to produce smooth motions that account for the captured geometry...
Weiming Zhi, Ziyong Ma, Tianyi Zhang et al.· Neural Information Processin...· 0 citations
D, a reference-guided renderer that extends Wan2.2 camera control from Plucker rays alone to a joint camera-plus-geometry interface and projects a neural 4D G-buffer from the animated mesh and injects it through a widened control adapter while preserving the pretrained image-to-video prior, supporting tracking+world-po...
Junhao Chen, Mingjin Chen, Henghaofan Zhang et al.· 0 citations
This work shows that camera motion, object trajectories, and depth can be unified into a single 3D point-track representation, from which one model performs joint camera and object control, depth editing, and motion transfer in a single forward pass, enabling interactive 4D-controllable streaming generation for the fir...
Shiqian Li, Chenguo Lin, Zhi-Guang Liu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
