Jul 2026· International Conference on Control, Decision and Information Technologies· pp. 1299-1304· 0 citations· 16 references
Abstract
Person search in real-world surveillance and robotic perception often fails when facial cues are unreliable due to low resolution, motion blur, or occlusion. We propose a clothing-centric person search framework that represents people using appearance attributes (e.g., garment type, dominant colors, accessories) instead of biometrics. Given person tracklets, the system samples frames, extracts clothing cues, and generates schema-constrained structured descriptions using a large language model, enabling consistent semantic indexing. Retrieval is performed with attribute-aware semantic search over these descriptions and compared against appearance-based baselines in controlled multi-camera experiments. A full-dataset analysis of the generated descriptions highlights common failure modes under real-world conditions, including color ambiguity under lighting changes (e.g., black vs dark), posture hallucinations under occlusion (e.g., standing behind a table described as sitting), and occasional prompt deviations that introduce spurious attributes. Runtime results show that detection is lightweight (about 30 ms per frame), while semantic extraction dominates; with parallel workers, the full pipeline processes roughly 0.25-0.34 s of computation per second of video, supporting practical deployment.
A mask-aware tri-modal framework that improves the quality of superpoint representations by retrieving a scene-level structural context from a pretrained PointSAM encoder to enhance object-centric evidence and predicting a soft mask weight to suppress unreliable superpoints.
Feng Zhou, Hui Wang, Kaida Ning et al.· The Visual Computer· 0 citations
A Multi-level Semantic-Guided (MSG) framework that integrates contextual and fine-grained visual information to eliminate clothing variance across both conceptual and pixel dimensions is proposed.
Shijuan Huang, Hefei Ling, Zongyi Li et al.· ACM Transactions on Multimed...· 0 citations
Person search is challenging due to limitations in identity representation. Existing methods rely on one-hot encoding, ignoring semantic relationships among pedestrians. This leads to a fragmented feature space and reduces generalization ability, especially in large-scale scenarios with a significant proportion of unlabeled identities. For instance, in the CUHK-SYSU dataset, 72.7% of pedestrians lack identity annotations, limiting the effectiveness of supervised learning. To address these issues, we propose a novel Multimodal-Guided Self-Distillation (MGSD) method for Unified Person Search that leverages multimodal textual descriptions and self-distillation to enhance pedestrian representation learning. Specifically, we introduce three key innovations: (1) Multimodal LLM-Assisted Text Generation (MLTG) to provide fine-grained semantic context beyond discrete identity labels, enabling the model to capture inter-person relationships based on clothing attributes, appearance features, and environmental cues; (2) Semantic Structural Consistency Constraint (SSCC) to impose global structural constraints on the feature space, ensuring that distinct identities remain separable while preserving semantic similarities among visually similar individuals; and (3) Multimodal-Aware Self-Distillation Framework (MSDF), where the learnable visual encoder is progressively aligned with the pre-trained CLIP multimodal encoder, improving robustness to variations in illumination, occlusion, and background clutter. Extensive experiments demonstrate that our method significantly enhances retrieval accuracy and generalization, achieving state-of-the-art performance with an mAP of 56.1% on the PRW dataset while maintaining computational efficiency for large-scale real-world applications.
Xi Yang, He-Xun Zhou, Hai-Yang Zhu et al.· Neural Networks· 0 citations
In real-world surveillance, situations arise in which two people wear nearly identical clothing or in which identifying features are obscured by heavy occlusions and shifting poses. In these environments, traditional uni-modal systems that rely on static appearance do not perform well and often produce false matches. This paper presents a multi-modal Re-ID framework that combines appearance, facial features, and gait to accurately perform this task. In the first stage, robust person tracklets are extracted from video using a YOLOv8 detector and a DeepSORT tracker. A multi-modal feature extractor is then applied to each person crop through three parallel streams. A custom GaitSet-inspired network produces a 256-dimensional gait embedding from RGB tracklet frames. An OSNet backbone is used to get a 1024-dimensional omni-scale appearance embedding. For facial biometrics, a Multi-Task Cascaded Convolutional Network (MTCNN) is used to explicitly detect the face and five-point landmarks followed by a facial landmark regression model. Each crop is aligned to a landmark and the aligned face is sent to FaceNet for computing the 128 dimensional facial embedding. In the last step, an Ensemble Concatenation module merges these three normalised embeddings into a single
$$\mathbb {R}^{1408}$$
identity descriptor (
$$1024 + 256 + 128$$
). The discriminability is significantly improved through the combination of facial features and temporal motion cues. Furthermore, the fusion strategy prevents any single modality from causing identity leakage, thereby providing a strong defense against appearance-based identity errors.
