This work proposes ViCo-SAM3, a Vision-Conditioned alignment framework designed for OVCOS, which introduces vision-conditioned (ViCo) module, which dynamically modulates text embeddings with global visual context, enabling textual representations to adapt to the current image content and thereby effectively bridges the semantic gap between vision and text.
Abstract
Open-vocabulary camouflaged object segmentation (OVCOS) aims to segment unseen camouflaged objects under text guidance. We observe that SAM3 still suffers from a pronounced semantic gap between global textual semantics and fine-grained pixel-level visual cues in OVCOS. Meanwhile, fully fine-tuning the text encoder introduces heavy parameter overhead and risks overfitting to training categories, which compromises open-vocabulary representation flexibility. To address these issues, we propose ViCo-SAM3, a Vision-Conditioned alignment framework designed for OVCOS. Specifically, we introduce vision-conditioned (ViCo) module, which dynamically modulates text embeddings with global visual context, enabling textual representations to adapt to the current image content and thereby effectively bridging the semantic gap between vision and text. Building on this, we further design a vision-conditioned cross-modal binding (ViCoBind) module to enhance cross-modal interaction and semantic alignment between visual and textual representations. Without bells and whistles, ViCo-SAM3 achieves state-of-the-art performance on the OVCamo benchmark and demonstrates strong generalization.
MODdapter extracts visual semantic cues from coarse segmentation results and integrates them as supplementary textual data, allowing for a more accurate projection of text features into the feature space and enhancing fine-grained recognition of unseen objects.
Jia-Xiang Fang, Shi-Qiang Ma, Jing Wang et al.· Neural Networks· 0 citations
This paper introduces a framework that modulates static textual embeddings into adaptable semantic guidance for RGB-D open-vocabulary 6D pose estimation, and designs a Semantic-Anchored Cross-view Fusion module that aggregates image features from the reference and query views using modulated semantics as an invariant a...
Zixuan Sun, H. Shuai, Qing-Shan Liu· IJCAI/ECAI· 0 citations
A new ITM framework that improves the model's discriminative performance by focusing on localized core attributes and has a better robustness in handling highly similar hard negatives, which provides a new way to address cross-modal hard sample discrimination.
Ya-Ting Liu, Kuang-Rong Hao· International Conference on...· 0 citations
This work developed a mask generator based on the denoising UNet from a pre-trained diffusion model, leveraging its capability for precise textual control over dense pixel representations and enhancing the open-world adaptability of the generated masks, and introduces a 2D-to-3D mask adaptation module to improve the qu...
Ziyi Wang, Yan-Bo Wang, Xu-Min Yu et al.· IEEE Transactions on Pattern...· 0 citations
This work proposes Language-Aligned Dense perception for COD (LAD-COD), a framework that aligns top-down semantic target guidance with bottom-up hierarchical visual features and allows semantic information to guide localization while preserving the fine structural details needed for camouflage segmentation.
This work addresses the challenge of open-vocabulary instance segmentation and open-set panoptic segmentation, which aim to recognize both predefined and unseen object categories without exhaustive human annotations, with a multimodal framework that leverages pre-trained vision-language models for automatic pseudo-labe...