Person search is challenging due to limitations in identity representation. Existing methods rely on one-hot encoding, ignoring semantic relationships among pedestrians. This leads to a fragmented feature space and reduces generalization ability, especially in large-scale scenarios with a significant proportion of unlabeled identities. For instance, in the CUHK-SYSU dataset, 72.7% of pedestrians lack identity annotations, limiting the effectiveness of supervised learning. To address these issues, we propose a novel Multimodal-Guided Self-Distillation (MGSD) method for Unified Person Search that leverages multimodal textual descriptions and self-distillation to enhance pedestrian representation learning. Specifically, we introduce three key innovations: (1) Multimodal LLM-Assisted Text Generation (MLTG) to provide fine-grained semantic context beyond discrete identity labels, enabling the model to capture inter-person relationships based on clothing attributes, appearance features, and environmental cues; (2) Semantic Structural Consistency Constraint (SSCC) to impose global structural constraints on the feature space, ensuring that distinct identities remain separable while preserving semantic similarities among visually similar individuals; and (3) Multimodal-Aware Self-Distillation Framework (MSDF), where the learnable visual encoder is progressively aligned with the pre-trained CLIP multimodal encoder, improving robustness to variations in illumination, occlusion, and background clutter. Extensive experiments demonstrate that our method significantly enhances retrieval accuracy and generalization, achieving state-of-the-art performance with an mAP of 56.1% on the PRW dataset while maintaining computational efficiency for large-scale real-world applications.
Xi Yang, He-Xun Zhou, Hai-Yang Zhu et al.· Neural Networks· 0 citations
In semi-supervised object detection (SSOD), due to the limited availability of labeled data, the quality and quantity of pseudo labels generated from unlabeled images are crucial for model training. Our study reveals that in the early stages of training, the number of usable pseudo labels is very low, which hampers the model's ability to converge quickly. Furthermore, throughout the training process, even though pseudo-labels are generated with high confidence, a significant number of labeling errors still occur. To address these challenges, we propose a novel framework, Generation-to-Optimization of Pseudo Labels (GO-PL), which comprehensively improves the model's performance in both the pseudo labels generation stage and the pseudo labels optimization stage. Specifically, we design a CLIP-based Pseudo Label Generation (CPLG) module, which matches proposals from the teacher model with CLIP text embeddings to generate semantically consistent high-confidence pseudo labels. These labels are further integrated into student training via an auxiliary loss, enabling early-stage guidance with stable and informative supervision signals. We also introduce Pixel-guided Instance Weighting (PIW) module that utilizes the SAM to assess the pixel-level quality of pseudo labels generated by the teacher model. Combined with a local voting mechanism, PIW dynamically assigns weights to each pseudo-label, thereby optimizing the impact of labels with varying quality on gradient updates. GO-PL demonstrates strong performance on standard SSOD benchmarks, including the MS-COCO and PASCAL VOC datasets. Notably, it achieves 26.7% mAP with only 1% labeled data on the MS-COCO, surpassing the previous best approach by 1.7%.
Xi Yang, Penghui Li, Nannan Wang· IEEE Transactions on Image P...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
