A CLIP-Guided Progressive Body-Part Semantic Alignment Network is proposed, termed PBSA-Net, which introduces CLIP-derived textual semantics as modality-agnostic guidance for both global representation learning and local body-part feature extraction.
Abstract
Visible-infrared person re-identification (VI-ReID) aims to retrieve pedestrian images of the same identity across visible and infrared modalities, but remains challenging due to the large modality gap and unstable local correspondence. Existing methods mainly rely on visual cues, which may be insufficient when infrared images lack color and fine-grained texture information. To address this issue, this paper proposes a CLIP-Guided Progressive Body-Part Semantic Alignment Network, termed PBSA-Net. The proposed method introduces CLIP-derived textual semantics as modality-agnostic guidance for both global representation learning and local body-part feature extraction. Specifically, a global semantic branch first learns identity-level textual anchors to regularize global visual features. Then, a body-part semantic branch exploits identity-aware body-part prompt learning, multi-level feature fusion, and text-guided cross-attention to guide fine-grained local representation learning. A progressive three-stage optimization strategy is further adopted to decouple global semantic learning, body-part semantic correspondence learning, and retrieval-oriented feature optimization. Experiments on SYSU-MM01, RegDB, and LLCM demonstrate the effectiveness of PBSA-Net. It achieves 76.5% Rank-1 and 74.2% mAP on SYSU-MM01, 82.5% Rank-1 and 76.0% mAP on RegDB, and 61.8% Rank-1 and 65.8% mAP on LLCM. Ablation studies further show that the proposed body-part semantic alignment and progressive optimization provide complementary improvements.
Visible-infrared person re-identification (VI-ReID) remains challenging due to severe spectral discrepancy and local cross-modal misalignment between visible and infrared images. Most existing methods alleviate this discrepancy through middle- or late-stage feature alignment, but modality-specific shallow features may have already accumulated spectral bias and local correspondence errors before shared representations are formed. In this paper, we argue that VI-ReID should be treated as an early cross-modal correspondence learning problem rather than only a late embedding alignment problem. To this end, we propose \textbf{CMIA-Net}, a framework that establishes bidirectional visible-infrared interaction at shallow backbone stages. Its core module, \textbf{Cross-Modal Interaction Attention (CMIA)}, enables visible and infrared feature maps to exchange complementary local information before deep semantic aggregation, thereby reducing progressive stream divergence. To stabilize early interaction, we further introduce \textbf{Spectral-Invariant Augmentation (MC-Aug)} to suppress over-reliance on visible-spectrum cues and a \textbf{Bi-directional Hetero-Center Learning (BHCT)} loss to improve class-center-level cross-modal compactness and inter-class separability. Experiments on RegDB and SYSU-MM01 show that CMIA-Net achieves strong performance on RegDB and the SYSU-MM01 indoor-search protocol, while remaining competitive under the more challenging SYSU-MM01 all-search setting. Specifically, CMIA-Net obtains 94.09\% Rank-1 accuracy and 90.74\% mAP in the RegDB visible-to-thermal setting, and 66.67\% Rank-1 accuracy and 58.12\% mAP in the SYSU-MM01 all-search setting.
Unknown authors· Engineering Research Express· 0 citations
Structural-Semantic Reciprocal Learning (SSRL), a framework that transforms open-loop association into a self-correcting closed-loop system, achieves robust cross-modal representation through the reciprocal interaction between structural and semantic learning.
Moyao Tian, Shijia Liu, Yan Yang et al.· arXiv.org· 0 citations
A Multi-level Semantic-Guided (MSG) framework that integrates contextual and fine-grained visual information to eliminate clothing variance across both conceptual and pixel dimensions is proposed.
Shijuan Huang, Hefei Ling, Zongyi Li et al.· ACM Transactions on Multimed...· 0 citations
A Progressively Biased Split Vision Transformer (PBSVT) is proposed, which combines a split ViT backbone with progressive bias training to gradually reduce RGB-dominant bias while preserving modality-shared structure and demonstrates the effectiveness of progressive modality transition for robust VI-ReID representation learning.
Mengru Jiao, Xin-Yue Xu, Junfeng Zhang· International journal of pat...· 0 citations
This work proposes CLIP-SGI, a semantic-guided and instance-consistent framework for generalizable person ReID that combines semantic guidance, domain-aware representation learning, and instance consistency to improve robustness under domain shifts.
Dai-Xin Liu, Yu Yang, Linlin Tang et al.· IEEE Transactions on Image P...· 0 citations
Visible-infrared person re-identification (VI-ReID) is an important technique for around-the-clock person matching, and its primary challenge arises from substantial cross-modal discrepancies. To address this challenge, we propose a Decoupled Information-Guided Cross-Modal Alignment (DIGCA) framework organized into three stages: multi-scale feature modeling, partial functional decoupling, and guided cross-modal alignment. First, a Hierarchical Context Extractor (HCE) aggregates multi-scale contextual information through dilated convolutions and residual connections to enrich the initial identity representation. Second, a Multi-Branch Unified Encoder (MBUE) organizes complementary feature streams through parallel global-relation and local-spatial modeling. The resulting representations exhibit different information tendencies and promote partial functional decoupling between identity-related and modality-related information. Finally, the Decoupled Information-Guided Cross-Modal Alignment (DIG-CMA) module refines the two streams with channel and spatial attention and integrates them through cross-attention. Under the joint optimization objective, the resulting representations support cross-modal alignment. Experiments on SYSU-MM01, RegDB, and LLCM show that DIGCA achieves competitive overall performance compared with recent VI-ReID methods, providing empirical support for the effectiveness of the proposed decoupling-guided alignment strategy in cross-modal identity matching.
