Skip to content

EOVSAM: Efficient Open-Vocabulary Segmentation with SAM 3 in One Pass

Aug 2026 · arXiv.org · Vol abs/2608.02284 · 0 citations · 35 references
Computer Science

TL;DR

An Efficient Open-Vocabulary segmentation framework with SAM 3 (EOVSAM), which adapts SAM 3 for single-pass prediction and introduces a new Attentional Aggregation strategy to optimize open-vocabulary classification end-to-end.

Abstract

Open-vocabulary segmentation identifies and segments objects from arbitrary textual descriptions. SAM 3 supports noun-phrase-guided segmentation and achieves competitive open-vocabulary performance through exhaustive vocabulary traversal, yet suffers from prohibitive computational overhead as target categories scale. In this paper, we propose an Efficient Open-Vocabulary segmentation framework with SAM 3 (EOVSAM), which adapts SAM 3 for single-pass prediction. EOVSAM removes prompt conditioning to turn SAM 3 into an efficient mask generator and introduces a new Attentional Aggregation strategy to optimize open-vocabulary classification end-to-end. This formulation avoids the multi-stage pipelines and post-processing heuristics commonly used by existing methods, while mitigating the closed-set collapse that can arise when classification is optimized directly. EOVSAM consistently improves segmentation accuracy over vanilla SAM 3 on all evaluated datasets and accelerates inference by up to 338$\times$. Furthermore, EOVSAM maintains high accuracy at lower resolutions while achieving even more remarkable inference speeds. Experiments on standard semantic and panoptic segmentation benchmarks show that EOVSAM combines competitive or state-of-the-art accuracy with a substantial speed advantage over existing open-vocabulary segmentation models. Code and models are available at https://github.com/hustvl/EOVSAM.

View source

Similar papers

Preprint Oct 2026

Efficient SAM3 Adaptation for Multi-Class Semantic Segmentation via Dense Competitive Representations

The Segment Anything Model 3 (SAM3) has achieved significant progress in Promptable Concept Segmentation (PCS) by processing short noun phrases to generate segmentation masks with unique instance identifiers. While SAM3 excels at single-concept segmentation, it falls short in complex, real-world environments where mult...

Wentinn Liao, Hao Zhu, Yike Ma et al. · 0 citations
Book Open access Nov 2026

Vocabulary-Free Segmentation via Multi-mask Category Generation with MLLMs

Mask-based recognition with multimodal large language models (MLLMs) offers a promising route to vocabulary-free segmentation. However, existing methods typically predict each candidate mask independently, limiting shared-context reasoning, while naive joint multi-mask generation suffers from cross-mask interference. W...

Tao Wang, Gui-Ping Cao, Yi-Peng Zhang et al. · 0 citations
Preprint Oct 2026

LLMSeg: Training-Free Text-Prompted Image Segmentation with B-Splines

Promptable segmentation models such as SAM 3 achieve remarkable open-vocabulary performance but require dedicated architectures, large-scale annotation engines, and task-specific training. We ask a different question: can a general-purpose multimodal large language model (MLLM) segment images directly, without an added...

Abdullah Hamdi · 0 citations
Review Nov 2026

TerraMesh-Masks: Open-Vocabulary Segmentation for Earth Observation

Open-vocabulary segmentation (OVS) enables models to segment a variety of concepts specified in natural language, removing the need for fixed label sets and fine-tuning. While recent OVS approaches have shown strong performance on natural images, they transfer poorly to Earth observation (EO) data due to differing sema...

Benedikt Blumenstiel, Hugues Devimeux, Johannes Jakubik et al. · 0 citations
Sep 2026

Preference-Guided Adaptation for Open-Vocabulary Semantic Segmentation via Prompt Disagreement

Open-vocabulary semantic segmentation (OVSS) enables pixel-level prediction over arbitrary text-specified vocabularies and has shown strong generalization on common benchmarks. However, OVSS performance often degrades in specialized domains such as medical imaging, remote sensing, and industrial inspection, where dense...

Hyun-Kurl Jang, Jihun Kim, Kuk-Jin Yoon · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.