Skip to content
Preprint

4D Reconstruction with Events&RGB

Oct 2026 · 0 citations · 25 references
Computer Science

TL;DR

It is suggested that even a lightweight, non-invasive fusion of event data can meaningfully improve 4D reconstruction robustness, motivating a more thorough architectural investigation in future work.

Abstract

Reconstructing dynamic 3D scenes over time (4D reconstruction) from a single moving camera is central to robotics and AR/VR applications, yet state-of the-art feed-forward methods rely solely on RGB frames and therefore inherit the failure modes of conventional cameras, including motion blur and temporal aliasing under fast or non-uniform camera and object motion. Event cameras report per-pixel brightness changes asynchronously at microsecond resolution, offering a complementary signal that is largely immune to these artifacts, but whether this modality can be incorporated into existing RGB based 4D reconstruction pipelines without a full architectural redesign remains an open question. In this preliminary study, we investigate a minimal extension: we augment Monst3R, an existing RGB-only 4D reconstruction model, with an additional event-camera input branch, leaving the rest of the architecture unchanged. Across 3 benchmark datasets, this simple RGB-event fusion reduces Abs Rel by 3.4% on average relative to the RGB-only Monst3R baseline, and a sensitivity analysis shows that the improvement grows with the magnitude of camera and object motion, with the largest gains observed under high object motion conditions where RGB-only reconstruction degrades most. These early results suggest that even a lightweight, non-invasive fusion of event data can meaningfully improve 4D reconstruction robustness, motivating a more thorough architectural investigation in future work.

View source

Similar papers

Preprint Nov 2026

Camera3R: Feed-Forward 3D Reconstruction from Heterogeneous Camera Models

Feed-forward 3D reconstruction (3R) from unconstrained multi-view images has recently emerged as a prominent research direction in 3D computer vision. Various foundational reconstruction models have been proposed with different architectures and reconstruction paradigms. Despite being trained on large-scale perspective...

R. Duan, Yun-Wen Zhou, Erin Hong et al. · 0 citations
Preprint Oct 2026

ARROW: Arbitrary Reconstruction and Tracking of 4D Observations in the Wild

Dynamic scenes may be captured by a moving camera, multiple video streams, or images taken at different times. These observations reveal complementary aspects of scene geometry and motion, yet bringing them together requires establishing correspondence across viewpoints, capture times, and visibility changes. We introd...

Ilya Fradlin, Christian Schmidt, Jens Piekenbrinck et al. · 1 citation
Preprint Nov 2026

Multi-View 3D Reconstruction Using V-JEPA Features

Predictive video encoders provide dense visual representations, but adapting them to 3D reconstruction requires accurate depth, camera poses, and cross-view geometry. We study supervised RGB-only multi-view reconstruction using an architecture that combines video-pretrained features, camera estimation, scene aggregatio...

Anwar Shamim, Jeffrin Sam · 0 citations
Sep 2026

OTT3R: Multi-View 3D Reconstruction and Fast Dataset Generation at 1% Compute

Feed-forward 3D reconstruction models have achieved impressive performance by scaling model and dataset size, but their cost excludes most research groups and precludes edge deployment. Additionally, generating 3D supervision without sensors still relies on slow, unreliable Structure-from-Motion, as the community lacks...

Brandon Leblanc, Charalambos Poullis · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.