Skip to content

Hybrid Gaussians for Robust Open-Vocabulary 3D Segmentation with Multi-View Object Association and Boundary Refinement

Sep 2026 · arXiv.org · Vol abs/2609.27462 · 0 citations · 27 references
Computer Science

TL;DR

This work introduces Hybrid Gaussians, a unified 3D representation jointly modeling object association and language-aligned semantics and its Multi-View Object Association mechanism combines Observation Fusion and Semantic Contrastive Learning to improve identity consistency and semantic discrimination.

Abstract

Open-vocabulary 3D segmentation localizes objects from free-form text queries, but remains challenging in real image sequences: incomplete or noisy 2D supervision destabilizes multi-view identity assignment, while full-scene semantic learning weakens object-level discriminability. We introduce Hybrid Gaussians, a unified 3D representation jointly modeling object association and language-aligned semantics. Its Multi-View Object Association mechanism combines Observation Fusion and Semantic Contrastive Learning to improve identity consistency and semantic discrimination. Boundary Reconstruction Optimization further refines local boundary structure to improve contour quality. Experiments on LERF and 3D-OVS demonstrate strong quantitative and qualitative performance. Our method achieves 59.1\% mIoU on LERF, yielding a 13.4\% relative gain over the baseline. Project page: https://nora202.github.io/hybridgaussians.

View source

Similar papers

Sep 2026

SenseFuse: Label-Free Fusion of Image and Shape Encoders for Open-Vocabulary 3D Instance Segmentation

Open-vocabulary scene understanding is fundamental for robotics, laying the groundwork for spatial reasoning and object manipulation. While closed-vocabulary 3D instance segmentation heavily leverages 3D shape information, state-of-the-art open-vocabulary methods remain predominantly restricted to 2D image features or...

Euiseok Han, Tri Ton, Hwanhee Kim et al. · 0 citations
Open access Sep 2026

Semantic-Guided Adaptive Gaussian Segmentation

3D Gaussian Splatting (3DGS) enables real-time photorealistic scene reconstruction, yet its segmentation tasks suffer from two critical flaws: poor 3D consistency (e.g., blurred instance boundaries and unstable cross-view semantic association) and insufficient structural awareness near ambiguous object boundaries. To a...

Ying-Han Zhou, Fan Zhou · 0 citations
Preprint Nov 2026

RefineSplat: Reconstruction-Guided Object-Centric Gaussian Splatting for Open-Vocabulary Querying

Open-vocabulary 3D scene understanding aims to segment and retrieve arbitrary objects in 3D scenes using natural language queries. Most existing methods typically follow a splat-then-segment paradigm, where 3D Gaussian Splatting is first reconstructed and then augmented with labels or language features for subsequent s...

Yiren Lu, Di-Sheng Liu, Sreya Srinidhi et al. · 0 citations
Preprint Oct 2026

Query-Time Calibration of CLIP Features for Open-Vocabulary 3D Segmentation

Lifting vision-language features such as CLIP into 3D Gaussian reconstructions enables open-vocabulary segmentation through natural-language queries. Although CLIP represents a broad range of semantic concepts, the lifted features of individual scenes can exhibit strong directional concentration, while raw cosine simil...

D. Engel, Yousef Rajeh, Leon Sick et al. · 0 citations
Preprint Oct 2026

DIGS: Distribution-Informed Gaussian Splatting for Training-Free Open-Vocabulary 3D Segmentation

Open-vocabulary 3D Gaussian segmentation typically binds semantic features to the per-Gaussian state via per-scene gradient training (1-4 h) or the more recent closed-form distillation. Both commit each primitive to a single feature; therefore, boundary Gaussians spanning multiple 2D instances are forced into a hard de...

Zhi-Fan Zeng, Wen-Bo Xiao, A. Sowmya et al. · 0 citations

EviSplat: Preserving Multi-View Evidence in 3D Gaussian Splatting for Open-Vocabulary Segmentation

EviSplat is introduced, which preserves individual observation features as evidence for later text queries within class-agnostic 3D instances that represent objects, object parts, or background regions and learns a distribution describing which visual appearances its observations support.

Sung-Ho Moon, Kota Shimomura, Junwoo Park et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.