Jul 2026· Annual International ACM SIGIR Conference on Research and Development in Information Retrieval· pp. 494-504· 0 citations· 21 references
Computer Science
TL;DR
A PETL-based framework named UniGR is presented that unifies granularity and reliability for robust and efficient TPR, and a multi-granularity relational adapter (MRA) is designed to capture both coarse-grained global and fine-grained local relational features among tokens.
Abstract
Text-based person retrieval (TPR) has become a crucial task in cross-modal retrieval due to its broad application in fields such as public safety and criminal investigation. Existing TPR methods typically rely on fully fine-tuning large-scale pretrained vision-language models like CLIP, which incurs high computational costs and tends to exhibit poor generalization in unseen domains due to overfitting. Fortunately, Parameter-Efficient Transfer Learning (PETL) has emerged as a lightweight alternative. However, applying PETL to TPR remains challenging, as its limited adaptation capacity struggles to capture intricate identity cues and becomes highly susceptible to gradient interference from unreliable image-text pairs. To address these challenges, we present a PETL-based framework named UniGR that unifies granularity and reliability for robust and efficient TPR. Specifically, we design a multi-granularity relational adapter (MRA) to capture both coarse-grained global and fine-grained local relational features among tokens, equipping the generic backbone with the task-specific, precise understanding needed for TPR. To combat the noise sensitivity of PETL, a reliability-aware reweighting strategy (RRS) is introduced to adaptively down-weight unreliable samples during training. Furthermore, we propose a parameter-free cross-modal cyclic verification (CMCV) module to mitigate ambiguities in cross-modal matching computations and refine retrieval ranking further. Experiments on benchmarks corroborate the superiority of UniGR among parameter-efficient methods. Remarkably, with only 4.5% of trainable parameters, UniGR outperforms most fully fine-tuned methods while maintaining strong generalization.
Text-Based Person Retrieval (TBPR) aims to locate a person in an image database based on a natural language description. While effective in theory, TBPR faces substantial challenges in real-world scenarios due to noisy correspondences—misaligned or weakly related image-text pairs—that significantly degrade retrieval performance. Existing methods often overemphasize hard negative mining, which inadvertently magnifies the impact of such noise. To address this issue, we propose Dynamic Uncertainty with Noisy Correspondences (DUNC), a novel framework that incorporates two key components: (1) Cross-modal Evidential Learning (CEL), which models bidirectional alignment uncertainty using a Dirichlet distribution to capture the confidence in image-text similarity, and (2) Dynamic Robust Loss (DRL), which adaptively selects and aggregates hard negative samples to reduce the influence of noisy instances and improve model robustness. Unlike conventional global-alignment approaches, DUNC exploits fine-grained local correspondences to enhance semantic alignment between modalities. By integrating uncertainty-aware modeling and adaptive contrastive supervision, our method is capable of effectively disentangling noisy from reliable training pairs. Extensive experiments conducted on three benchmark datasets—CUHK-PEDES, ICFG-PEDES, and RSTPReid—demonstrate that DUNC consistently achieves state-of-the-art performance and exhibits strong robustness across a wide range of noise conditions. Code is publicly available at https://github.com/ASL-forever/DUNC.
Zequn Xie, Chuxin Wang, Sihang Cai et al.· ACM Transactions on Informat...· 0 citations
The Semantic-Aware Fine-Tuning (SAFT) framework is proposed to address semantic compression in specific domains, which incorporates Semantic-Aware Soft-Label Supervision and Intra-modal Structural Distillation to establish a promising paradigm for domain-specific TBIR tasks.
Jingyang Tan, Shengan Yang, Yuanpeng Chen et al.· 0 citations
Text-based person anomaly search requires retrieving real-world pedestrian images from detailed natural-language descriptions using models trained primarily on synthetic data. This Sim2Real setting is particularly challenging because visually similar candidates may differ only in subtle actions, object interactions, or appearance attributes, while applying multimodal large language models to the entire gallery is computationally expensive. We propose an anchor-constrained coarse-to-fine retrieval framework that combines global semantic matching with fine-grained verification. First, each query is represented by its original caption, a structured concatenation, and several semantic facets. Heterogeneous vision-language retrievers are then integrated through robust per-query score calibration and soft claim-aware fusion. Full and concatenated captions serve as anchors to preserve candidate recall, whereas appearance, action, and object facets provide bounded corrective evidence. The resulting candidate pool is further refined by a discriminative Qwen3 reranker and two complementary semantic verification modules based on anomaly-aware cloze completion and multi-agent evidence reasoning. Finally, an uncertainty-gated consensus module adaptively reweights the three experts on ambiguous queries. Experiments on the PAB benchmark show that the proposed soft claim-aware retrieval achieves 86.44% mAP@10, substantially outperforming individual retrieval backbones. The complete framework further improves performance to 95.41% mAP@10, 94.44% R@1, and 99.09% R@5. These results demonstrate that preserving strong global retrieval while restricting expensive semantic reasoning to a small candidate pool is effective for fine-grained Sim2Real person anomaly search. Our code will be available on Github.
H. Pham, P. Tran, Thuan Duc Mai et al.· 1 citation
Text-based person search (TBPS) aims to retrieve images of a target person from a large image gallery based on a given natural language description. Most existing methods rely on supervised learning with manually annotated image-text pairs. In this paper, we explore unsupervised TBPS, with only unlabeled images. We propose GTR$^+$, a two-stage generation-then-retrieval framework. In the generation stage, we introduce a tiered description generation framework designed to produce fine-grained and stylistically diverse textual descriptions through a three-tier sequential process. The base tier leverages an automated question-and-answer mechanism to generate basic visual attribute descriptions; the intermediate tier enhances fine-grained detail using an inter-sample contrastive mechanism; the advanced tier further enriches textual diversity via a stylized expansion mechanism. In the retrieval stage, to mitigate the impact of noisy pseudo texts, we develop an adaptive confidence-weighted retrieval learning framework. We model image-text pairs as clean or noisy using a Gaussian Mixture Model, calibrated by real-time image-text similarity and static text generation probability from the prior stage, yielding adaptive sample weights during training. Beyond that, we also contribute LargeFine-Person, a large-scale TBPS dataset with high-quality, fine-grained, and diverse textual annotations, enabling a practical and generalizable TBPS pre-training benchmark under unsupervised setting.
Mang Ye, Yucheng Ji, Yang Bai et al.· IEEE Transactions on Pattern...· 0 citations
