Jul 2026· Annual International ACM SIGIR Conference on Research and Development in Information Retrieval· pp. 5021-5025· 0 citations· 31 references
Computer Science
TL;DR
ReLSum is a reinforcement learning framework that generates concise, relevance-optimized product summaries for search ranking by using downstream relevance scores as reward signals and enables summaries to be generated and cached offline, ensuring no additional inference latency at serving time.
Abstract
E-commerce search ranking models face the challenging and critical problem of balancing strict real-time latency constraints with the need for high-quality relevance predictions. In production environments, ranking models often rely primarily on product titles, which frequently omit critical attributes required to satisfy diverse query intents. While full product descriptions provide richer information, their length and verbosity make them computationally impractical for real-time ranking, particularly when using cross-encoder architectures. To address this challenge, we propose ReLSum, a reinforcement learning framework that generates concise, relevance-optimized product summaries for search ranking. ReLSum directly aligns summarization with the ranking objective by using downstream relevance scores as reward signals. The framework conditions the Large Language Model (LLM) solely on product information, while queries are used only to compute rewards during training. This design enables summaries to be generated and cached offline, ensuring no additional inference latency at serving time. Experiments on large-scale production data show substantial improvements in offline NDCG and recall. In online A/B tests, ReLSum delivers statistically significant gains in user engagement metrics such as orders per visitor and units per completed order, with particularly strong improvements for tail queries.
Dense retrievers rank documents using vector similarity between a frozen encoder and a precomputed index. While test-time ranking rewards from a reranker or LLM judge can improve results, existing methods discard this signal after a single query. Updating the retriever's weights makes rewards reusable, but this requires parameter access, which is unavailable for closed-source models, and is computationally prohibitive. We propose TTT-Embed (Test-Time Tuning of Embeddings), a framework that distills ranking rewards into a lightweight, learned vector within the output embedding space of a frozen model. This vector is optimized purely from scalar ranking scores assigned to the retriever's own candidate documents, requiring no access to model weights, ground-truth labels, or modifications to index. A single scope parameter controls rewards reuse (global, task, or query), enabling a principled trade-off between reusability and specificity under a fixed reward computation budget. We demonstrate that as the available reward budget scales, the optimal sharing scope shifts dynamically from global-wise to task-wise and finally to query-wise. Evaluated across five embedding models and 15 MTEB retrieval tasks, TTT-Embed improves test-time retrieval by up to +8.36 nDCG@10. Crucially, the learned states generalize effectively to unseen queries (up to +8.57 nDCG@10) and unseen tasks (up to +4.71 nDCG@10). Furthermore, TTT-Embed successfully resolves catastrophic forgetting: by leaving base weights entirely frozen, it recovers degraded general capabilities (up to +8.00 nDCG@10, even surpassing the original base model) while preserving in-domain specialization. These results establish ranking rewards as a reusable test-time state, enabling budget-efficient adaptation for any embedding model, including closed-source APIs.
In large-scale e-commerce retrieval, dual-encoder retrievers are op- timized for contrastive similarity, whereas downstream rerankers capture finer-grained relevance preferences; this objective mis- match limits end-to-end retrieval quality. Reinforcement Learning offers a way to use reward-model feedback for retriever adaptation, but we observe that standard policy-gradient updates can degrade embedding geometry, especially when the document index must remain frozen due to industrial constraints. To address this, we propose PAO (Positive-Advantage-Only), a selective RL optimization method. Our analysis reveals that in- discriminate penalization of negative samples (pushing away) in a frozen high-dimensional space disrupts pre-trained semantic man- ifolds. PAO selectively applies gradient updates only to retrieved items with positive advantages, effectively pulling query embed- dings toward high-reward regions while preserving global topo- logical stability. Experiments on both a massive industrial dataset and public benchmarks demonstrate that PAO significantly outper- forms standard RL and distillation baselines.
Shaowei Wei, Chong Huang, Songtao Fang et al.· 0 citations
Multi-objective ranking serves as the backbone of industrial information retrieval, requiring a holistic assessment of documents across dimensions such as Relevance, Authority, and Recency. The prevailing industry paradigm relies on ensembles of specialized BERT-based models, which are costly to maintain and fundamentally limited in expressiveness by the modest model size of discriminative encoders. While Large Language Models (LLMs) offer a path to unified reasoning, their auto-regressive latency is prohibitive for real-time serving. In this paper, we propose UniRank, a Homogeneous Scaling framework that consolidates multiple ranking objectives into a single, unified generative model. To reconcile the conflict between deep reasoning and inference speed, we introduce two paradigmatic innovations: (1) Parallel Prediction via Task-Specific Anchors: We replace slow sequential decoding with a parallel mechanism that predicts all objective scores simultaneously in a single forward pass, achieving O(1) inference complexity. (2) Reinforcement Learning Optimization with Posterior CoT: We propose a novel training framework that synergizes Posterior Chain-of-Thought with GRPO. By forcing the model to generate rationales after scoring and aligning them via policy optimization, we effectively distill complex reasoning capabilities into the anchor tokens without incurring inference-time overhead. Extensive experiments on the MS MARCO benchmark and a massive industrial dataset from Tencent demonstrate that UniRank significantly outperforms BERT-Large and matches the performance of larger LLMs, all while maintaining strictly low latency. Online A/B testing further confirms its practical value, showing that UniRank simplifies the deployment pipeline while delivering a significant lift in core business metrics.
Dezhi Ye, Junwei Hu, Xiaoyang Chen et al.· Proceedings of the 32nd ACM...· 0 citations
Ranking is a fundamental component of modern information access systems. Reinforcement learning (RL) provides a flexible framework for directly optimizing coarse-grained feedback and system-level objectives defined over the complete ranking list. However, existing RL-based ranking methods typically treat each sampled permutation as an atomic output and evaluate it primarily through a scalar reward, overlooking the structural relationships among different ranking lists. Consequently, permutations with similar rewards but substantially different permutation patterns may receive comparable optimization signals, potentially leading to inaccurate credit assignment and overly aggressive policy updates. To address this limitation, we propose SRPO, a \textbf{S}tructure-aware \textbf{R}elative \textbf{P}olicy \textbf{O}ptimization framework for listwise ranking. SRPO measures the discrepancy between sampled permutations using a top-weighted Kendall-tau distance and normalizes their pairwise reward differences by the corresponding distances. It quantifies the reward improvement per unit of ranking change, thereby emphasizing efficient local refinements, particularly those involving top-ranked positions. Experimental results across two ranking scenarios demonstrate that explicitly modeling permutation-level differences improves the effectiveness and stability of listwise ranking, with particularly favorable performance in limited-feedback and complex list-level optimization settings.
Yiteng Tu, Weihang Su, Zitao Su et al.· arXiv.org· 0 citations
This work proposes a novel Token Selection approach for Preference Optimization in LLM-based sequential Recommendation, i.e., TSPORec, which accurately pinpoints informative tokens throughout the entire textual content to improve recommendation performance.
Wenqiao Zhu, Chao Xu, Haipang Wu et al.· 0 citations
STAR—Staged Training with Aligned Reinforcement Learning and Multi-Faceted Distillation is proposed, a progressive framework that follows a reasoning, ranking, and transfer pipeline to imbue dense models with both high performance and interpretability.
Chenxu Wang, Jianzhi Shao, Chi Zhang et al.· Annual International ACM SIG...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
