PaIR: Partition-Based Information Rebalancing for Robust Text-Based Person Search

Abstract

Text-based person search (TPS) suffers from cross-modal informational skewness: pedestrian images are high-dimensional and redundancy-prone, while textual descriptions are sparse, incomplete, and sometimes inaccurate. To address the low alignment accuracy and poor robustness caused by the inherent uneven information distribution of visual and textual modalities in TPS, this paper proposes a unified Partition-based Information Rebalancing (PaIR) framework to realize balanced optimization and precise alignment of cross-modal information from both global content and local part dimensions. The framework adopts the CLIP dual-modal encoder for basic feature extraction and constructs a parallel global–local dual representation system to compensate for the lack of fine-grained spatial information in single global features. To eliminate modal redundancy and noise interference, a dual-modal noise suppression module is designed to filter invalid redundant information through visual foreground–background separation and textual token weight screening, while introducing adversarial constraints and orthogonal constraints to purify effective features. On this basis, a part balance alignment module is built to complete human semantic part decomposition and soft matching alignment for dual-modal features. Aiming at the common part semantic missing problem in textual descriptions, a visual part correlation affinity matrix is utilized for semantic associative completion to balance the information density of dual modalities. Finally, a global–local joint alignment strategy integrates hierarchical features and bidirectional cross-modal attention interaction to eliminate global–local semantic discontinuity and enhance fine-grained cross-modal matching capability. Extensive experiments on three public benchmarks demonstrate that PaIR consistently improves multiple baselines.