Skip to content
Book Open access

RISE: Reconstructing Interactive 3D Scene from a Single Image

Dec 2026 · 0 citations

Abstract

Interactive 3D scenes serve as the foundational infrastructure for embodied AI, digital twins, and immersive metaverses, providing the essential physical and semantic substrate for realistic agent-environment interaction. While recent advances have made single-image 3D reconstruction increasingly feasible, existing methods still fail to directly reconstruct interactive scenes with part-level articulated structures for fine-grained physical interactions. The primary challenges lie in integrating fine-grained, interactable object modeling with holistic scene layouts, and in recovering detailed interactive properties from limited supervision. To bridge this gap, we introduce the task of Single-Image 3D Interactive Scene Reconstruction, which aims to synthesize physically interactive 3D scenes directly from a single image. By unifying fine-grained object articulation with holistic scene synthesis, this task enables the efficient derivation of simulation-ready environments from real-world observations, representing a pivotal step toward the scalable creation of interactive digital worlds. To solve the task, we propose RISE, a unified framework that extends the coarse-to-fine 3D reconstruction pipeline into an interactive paradigm. The coarse stage reconstructs rough object geometry and scene layout, while the fine stage refines object geometry and appearance. To infer interactive properties, we introduce a Generative Articulation Estimation (GAE) module between the coarse and fine stages. By extracting articulation parameters directly from compact coarse-stage representations, GAE efficiently recovers interactive properties while decoupling joint reasoning from entangled geometry. Furthermore, by reformulating articulation estimation as a 4D generative task, GAE leverages 3D priors to ensure geometric- and motion-aware predictions. To evaluate reconstructed interactive scenes, we further introduce three sets of metrics that comprehensively assess scene reconstruction and interactive property recovery. Experiments on both holistic interactive scene and single articulated object reconstruction demonstrate that our method generates high-quality interactable assets. We further validate that the interactive scenes reconstructed by our framework significantly expand the data scaling capacity for downstream embodied AI applications.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.