A modular mapping architecture is demonstrated that establishes 3D Semantic Scene Graphs (3DSSGs) as its foundational back-end, enabling the dense representation of extensive environments containing thousands of unique object instances and supporting open-vocabulary queries via CLIP features without requiring any additional post-processing steps.
A unified multi-axis taxonomy is introduced that enables us to classify the available methods in 3D Gaussian splatting methods in terms of five complementary categories: semantic vocabulary space, representation form, functional role, knowledge source, and query mechanism.
This work proposes a unified autoregressive diffusion-based graph generative model that jointly learns structure and features, constructing complete 3DSGs bottom-up from observed vertical planes across arbitrary hierarchy depths, and proposes an adaptation of the Fused Gromov--Wasserstein distance for principled graph-level evaluation of generated 3DSGs against ground truth.
J. A. Millan-Romera, Samuel Cognolato, Holger Voos et al.· 0 citations
Open vocabulary 3D scene understanding is essential for next-generation interactive systems, empowering users to intuitively query and navigate reconstructed environments using natural language. However, current 3D Gaussian frameworks are often bottlenecked by restrictive multiview capture requirements, costly scene-specific optimization, and the massive memory overhead of storing dense language features. We present a novel, training-free pipeline that fundamentally reimagines this paradigm by explicitly decoupling 3D geometric reconstruction from semantic integration. Given a standard monocular video sequence as input, our method efficiently outputs a compact, highly interpretable, and fully searchable object-level semantic Gaussian map. Rather than entangling heavy language embeddings within the mapping loop, we extract geometry independently and ground semantics through a lightweight, modular post-processing framework. Extensive evaluations on the Replica dataset demonstrate that this decoupled architecture preserves strong rendering fidelity and competitive segmentation accuracy. Crucially, by replacing dense per-Gaussian storage with modular, object-level semantic embeddings, our approach delivers an order-of-magnitude reduction in memory usage compared to SOTA baselines. This provides a highly efficient, scalable, and practical solution for open-vocabulary 3D retrieval and question answering directly from everyday monocular video.
Abstract. We demonstrate an end-to-end pipeline for 3D scene understanding which integrates unsupervised graph-based point cloud segmentation with LLM-enabled spatial reasoning and editing. A point cloud is segmented into a SemanticPatch decomposition (stage 1), labeled using a zero-shot vision-language model (stage 2; SAMv2, CLIP), encoded into a scene graph in the latent space (stage 3) capturing geometry, topology, and constraints, and finally manipulated by an LLM-based agent (stage 4) to execute a specified editing task. The LLM agent can be instructed by natural language input to reason about a scene graph and a point cloud, compute a geometric transformation for the input point cloud, and check its own output against a set of constraints (e.g. ADA-compliance). We validate our approach on three different point clouds: a classroom (Leica RTC360, 1.3 M points), a construction site (NavVis VLX mobile scanner, 4.4M points), and the Paris-Lille-3D benchmark. Our segmentation approach scores 97–99% on the fitness score and 92–99% on the F1-score across all three benchmarks. Our LLM agent solves reconfiguration tasks in 1–10 min, achieving a 100% constraint-satisfaction rate and outperforming a human annotator.
Florent Poux, Alex Key· The International Archives o...· 0 citations
A 3D Local- Global Linear Attention Mechanism (LG-LAM) is devised that efficiently captures long-range contextual information with linear complexity, enabling a comprehensive understanding of the 3D scene without heavy computational burdens.
Jie Li, Jiaheng Xu, Laiyan Ding et al.· International Conference on...· 0 citations
We study open-vocabulary 3D indoor layout generation, which synthesizes diverse and physically plausible scenes from unlabeled 3D assets using free-form language instructions. Recent methods leverage large language models (LLMs) and vision-language models (VLMs) to generate structured scenes from text. However, most model inter-asset relations implicitly or rely on local pairwise constraints and local optimization. These formulations are poorly aligned with the global, highly non-convex layout space, often yielding locally plausible yet globally inconsistent or physically infeasible scenes. We address this problem with a graph-based intermediate representation that separates semantic coherence from physical feasibility, together with a hybrid search-and-refinement strategy. First, Global Semantic Verification (GSV) represents scenes as structured graphs and enforces semantic constraints through rule-based verification. This explicit validation removes contradictory configurations and produces a globally consistent semantic scaffold. Second, Global Physical Feasibility Search (GPFS) combines evolutionary search for global exploration with gradient-based refinement for local exploitation. It reduces dependence on VLM-proposed initialization and improves robustness in non-convex and discontinuous feasible spaces. Together, GSV and GPFS move layout generation beyond local relational modeling and initialization-sensitive optimization toward globally consistent reasoning and search. Experiments show that our method achieves state-of-the-art performance in open-vocabulary 3D indoor layout generation, improving both semantic consistency and physical plausibility.
Jialu Huang, Yingxuan You, Fei Wang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
