This work introduces Hybrid Gaussians, a unified 3D representation jointly modeling object association and language-aligned semantics and its Multi-View Object Association mechanism combines Observation Fusion and Semantic Contrastive Learning to improve identity consistency and semantic discrimination.
Abstract
Open-vocabulary 3D segmentation localizes objects from free-form text queries, but remains challenging in real image sequences: incomplete or noisy 2D supervision destabilizes multi-view identity assignment, while full-scene semantic learning weakens object-level discriminability. We introduce Hybrid Gaussians, a unified 3D representation jointly modeling object association and language-aligned semantics. Its Multi-View Object Association mechanism combines Observation Fusion and Semantic Contrastive Learning to improve identity consistency and semantic discrimination. Boundary Reconstruction Optimization further refines local boundary structure to improve contour quality. Experiments on LERF and 3D-OVS demonstrate strong quantitative and qualitative performance. Our method achieves 59.1\% mIoU on LERF, yielding a 13.4\% relative gain over the baseline. Project page: https://nora202.github.io/hybridgaussians.
Open-vocabulary scene understanding is fundamental for robotics, laying the groundwork for spatial reasoning and object manipulation. While closed-vocabulary 3D instance segmentation heavily leverages 3D shape information, state-of-the-art open-vocabulary methods remain predominantly restricted to 2D image features or...
Euiseok Han, Tri Ton, Hwanhee Kim et al.· arXiv.org· 0 citations
3D Gaussian Splatting (3DGS) enables real-time photorealistic scene reconstruction, yet its segmentation tasks suffer from two critical flaws: poor 3D consistency (e.g., blurred instance boundaries and unstable cross-view semantic association) and insufficient structural awareness near ambiguous object boundaries. To a...
Ying-Han Zhou, Fan Zhou· Italian National Conference...· 0 citations
Open-vocabulary 3D scene understanding aims to segment and retrieve arbitrary objects in 3D scenes using natural language queries. Most existing methods typically follow a splat-then-segment paradigm, where 3D Gaussian Splatting is first reconstructed and then augmented with labels or language features for subsequent s...
Yiren Lu, Di-Sheng Liu, Sreya Srinidhi et al.· 0 citations
Lifting vision-language features such as CLIP into 3D Gaussian reconstructions enables open-vocabulary segmentation through natural-language queries. Although CLIP represents a broad range of semantic concepts, the lifted features of individual scenes can exhibit strong directional concentration, while raw cosine simil...
D. Engel, Yousef Rajeh, Leon Sick et al.· 0 citations
Open-vocabulary 3D Gaussian segmentation typically binds semantic features to the per-Gaussian state via per-scene gradient training (1-4 h) or the more recent closed-form distillation. Both commit each primitive to a single feature; therefore, boundary Gaussians spanning multiple 2D instances are forced into a hard de...
Zhi-Fan Zeng, Wen-Bo Xiao, A. Sowmya et al.· 0 citations
EviSplat is introduced, which preserves individual observation features as evidence for later text queries within class-agnostic 3D instances that represent objects, object parts, or background regions and learns a distribution describing which visual appearances its observations support.
Sung-Ho Moon, Kota Shimomura, Junwoo Park et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.