A Semantic-Retrieval-Augmented Detector (SRA-Det) is proposed that uses an attention-based module to retrieve multiple semantic facets from token-level text features, and a soft-min matching rule that behaves like a differentiable logical AND over these facets, ensuring that all key attributes are satisfied.
Experimental results show that the proposed open-vocabulary object detection framework performs excellently on multiple benchmark datasets such as LVIS and COCO-O, demonstrating stronger adaptability to complex scenarios.
Tao Liu, Chongwen Wang· International Conference on...· 0 citations
OVIP-SG is presented, a unified framework for instance-preserving semantic mapping, functional scene partitioning, and language-guided small, fine-grained object retrieval that outperforms ConceptGraphs under a unified evaluation protocol on Replica.
Tianjing Hao, Hai-Yu Lan, Ang Li et al.· 0 citations
Open-vocabulary object detection and segmentation aim to recognize arbitrary objects beyond predefined categories. Although recent vision-language and reference-based approaches have significantly advanced this field, they often rely on text prompts, limited visual examples, or expensive feature matching procedures, making them difficult to scale to large and continuously expanding object repositories. In this work, we propose VocaDet, a sample-driven open-vocabulary object detection and segmentation framework that learns object concepts directly from user-provided positive and negative sample collections without model retraining. The key idea is to transform continuous visual representations into discrete visual vocabularies and perform efficient retrieval-based recognition through a scalable vector database. Specifically, we employ DINOv3 as the visual feature extractor and apply agglomerative clustering with adaptive clustering sensitivity to generate multi-granularity visual tokens. These visual tokens, together with position-debiased representations and spatial topology information, are stored as expandable object memories in a vector database. During inference, query images are converted into visual tokens and efficiently matched against the stored object memories for object localization and segmentation. Furthermore, a background filtering mechanism is introduced to remove frequently occurring background patterns and reduce redundant retrieval operations in practical fixed-camera scenarios. Experiments on the UA-DETRAC dataset demonstrate that VocaDet achieves effective open-vocabulary detection performance without conventional detector training, while supporting continuously expandable recognition capability as additional positive and negative samples are accumulated.
Open-vocabulary object detection leverages vision-language pre-training to achieve cross-category generalization. With the progressive refinement of models such as GLIP, Grounding DINO, and YOLO-World, detectors have exhibited substantially enhanced zero-shot recognition capabilities on open vocabularies, with YOLO-World further advancing this capability into an efficient single-stage detection paradigm. However, during scenario-specific fine-tuning, training that relies solely on ground-truth annotations of base classes induces rapid degradation in the model’s discriminative ability for novel classes, often reverting to closed-set detector behavior. To address this issue, the present paper proposes a forgetting-resistant fine-tuning framework that decouples the training objectives into strong supervised adaptation for base classes and weak supervised preservation for novel classes. Specifically, novel pseudo-boxes are generated once by a frozen zero-shot teacher before any task-specific fine-tuning and are refined through threshold filtering, score weighting, and soft-ignore mechanisms to reduce the impact of noisy pseudo-supervision. Subsequently, teacher-guided logit distillation is incorporated, constraining the student model exclusively at the output distribution level to preserve its response relationships to the novel vocabulary. Experimental results demonstrate that conventional fine-tuning on YOLO-World-v2-S reduces Novel AP50 from 57.6 to 1.6. In contrast, through stepwise integration of the freezing strategy, pseudo-labels, and logit distillation, the proposed method elevates Novel AP50 to 51.6 and achieves an All AP50 of 53.7. These findings show that the proposed approach substantially mitigates catastrophic forgetting while preserving the original single-stage inference pipeline.
Recent unified open-vocabulary detection (OVD) supports heterogeneous prompts, including text queries, visual exemplars, and their combinations, but often rely on increasingly complex designs such as heavy cross-modal fusion, staged training, and iterative annotation pipelines. We revisit whether such complexity is necessary in the era of stronger foundation models. Our finding is that unified OVD can be made substantially simpler with semantic-rich visual representations and scalable grounding supervision. We present OPUS (\textbf{O}pen-vocabulary, \textbf{P}rompt-\textbf{U}nified, \textbf{S}imple), a unified detector supporting text, interactive visual, generic visual, and mixed prompting within one framework. OPUS adopts a simple three-part design. Its model architecture combines a semantic-rich visual encoder, built on a DINOv3-ConvNeXt-B backbone with efficient hybrid encoding, with a prompt-aware decoder that avoids prompt-specific branches for unified prompt reasoning. OPUS is trained with a one-stage text-visual training strategy with Instance-level Contrastive Alignment (ICA), and is supported by a SAM3-based single-pass data engine for heterogeneous grounding supervision. Experiments on COCO, LVIS-minival, and ODinW35 show that OPUS achieves state-of-the-art Visual-I performance, reaching 68.1/69.2/54.7 AP, while maintaining balanced Text and Visual-G accuracy. OPUS also turns mixed prompting from interference into complementarity, improving over text or visual prompt alone. These results show that simplicity and strong unified prompting capability can be achieved together.
Xiao-Yan Wei, Zhi-Min Yao, Rui-Lin Yang et al.· 0 citations
