Jul 2026· Annual International ACM SIGIR Conference on Research and Development in Information Retrieval· 0 citations· 24 references
Computer Science
TL;DR
STAR—Staged Training with Aligned Reinforcement Learning and Multi-Faceted Distillation is proposed, a progressive framework that follows a reasoning, ranking, and transfer pipeline to imbue dense models with both high performance and interpretability.
Abstract
E-commerce search relevance modeling faces a critical dilemma: traditional models falter with complex queries, while Large Language Models (LLMs), despite their superior reasoning, suffer from the prohibitive latency of auto-regressive Chain-of-Thought (CoT) generation, rendering them infeasible for production. Knowledge distillation offers a promising solution, yet current methods force an undesirable trade-off: sacrificing the very interpretability that makes LLMs powerful, or relying on expensive, unscalable human-annotated rationales. To address this, we propose STAR—Staged Training with Aligned Reinforcement Learning and Multi-Faceted Distillation, a progressive framework that follows a reasoning, ranking, and transfer pipeline to imbue dense models with both high performance and interpretability. First, STAR aligns a teacher LLM's reasoning with task objectives using a novel multi-granularity reward in Group Relative Policy Optimization (GRPO), leveraging only binary labels. Next, it refines the teacher's ability for calibrated scoring via token-level supervision, enabling efficient ranking through a single forward pass without any additional layers. Finally, this ''white-box'' knowledge is transferred to a compact student via multi-faceted distillation that preserves both reasoning logic and ranking behavior. Offline experiments demonstrate that our 0.6B student model rivals the performance of a strong 8B baseline, making it highly efficient and fully deployable. Real-world effectiveness is validated by significant online A/B test gains, including a +0.93% GoodRate lift and a +1.04% increase in GMV. STAR has been fully deployed to 100% of main search traffic on 1688.com.
I Reasoning via Multi-Teacher Distillation is proposed, a novel framework that 'compiles' the reasoning abilities of large teacher LLMs into a lightweight student Small Language Model (SLM), which significantly outperforms state-of-the-art baselines in both recommendation accuracy and inference efficiency.
Aspect-based sentiment analysis (ABSA) quadruple extraction requires jointly predicting target, aspect, opinion, and sentiment over reviews that often contain multiple fine-grained sentiment tuples. While large chain-of-thought (CoT) models perform well on this task, distilling them into smaller deployable models remains difficult. We identify a task-specific failure mode in distilled ABSA extraction: student errors at the target-aspect interface create structurally invalid states, such as broken target-aspect bindings and hallucinated targets, which then corrupt downstream predictions. Conventional off-policy distillation is poorly suited to this setting because it trains only on teacher-generated trajectories and provides little supervision on the student-induced structural states that dominate inference. To address this mismatch, we propose STAR-OPD (STructured Aspect-cascade-aware On-Policy Reward Distillation), which builds on generic on-policy distillation and instantiates it for ABSA quadruple extraction with cascade-aware, set-structured rewards. STAR-OPD trains on student rollouts and applies set-structured rewards that directly target binding consistency, target grounding, and fine-grained aspect disambiguation. Experiments on E-ABSA20K and SemEval-2014 show that STAR-OPD consistently outperforms off-policy and general on-policy baselines, reduces target hallucination, and substantially improves performance on structurally hard cases. With Qwen3-4B, STAR-OPD substantially narrows the student-teacher gap while improving inference efficiency, highlighting the importance of on-policy structural correction for distilled ABSA extraction.
ReLSum is a reinforcement learning framework that generates concise, relevance-optimized product summaries for search ranking by using downstream relevance scores as reward signals and enables summaries to be generated and cached offline, ensuring no additional inference latency at serving time.
Multi-objective ranking serves as the backbone of industrial information retrieval, requiring a holistic assessment of documents across dimensions such as Relevance, Authority, and Recency. The prevailing industry paradigm relies on ensembles of specialized BERT-based models, which are costly to maintain and fundamentally limited in expressiveness by the modest model size of discriminative encoders. While Large Language Models (LLMs) offer a path to unified reasoning, their auto-regressive latency is prohibitive for real-time serving. In this paper, we propose UniRank, a Homogeneous Scaling framework that consolidates multiple ranking objectives into a single, unified generative model. To reconcile the conflict between deep reasoning and inference speed, we introduce two paradigmatic innovations: (1) Parallel Prediction via Task-Specific Anchors: We replace slow sequential decoding with a parallel mechanism that predicts all objective scores simultaneously in a single forward pass, achieving O(1) inference complexity. (2) Reinforcement Learning Optimization with Posterior CoT: We propose a novel training framework that synergizes Posterior Chain-of-Thought with GRPO. By forcing the model to generate rationales after scoring and aligning them via policy optimization, we effectively distill complex reasoning capabilities into the anchor tokens without incurring inference-time overhead. Extensive experiments on the MS MARCO benchmark and a massive industrial dataset from Tencent demonstrate that UniRank significantly outperforms BERT-Large and matches the performance of larger LLMs, all while maintaining strictly low latency. Online A/B testing further confirms its practical value, showing that UniRank simplifies the deployment pipeline while delivering a significant lift in core business metrics.
Dezhi Ye, Junwei Hu, Xiaoyang Chen et al.· Proceedings of the 32nd ACM...· 0 citations
This work instantiates Feedback-Augmented Self-Distillation (FA-SD), a self-distillation algorithm for agentic search that leverages successful demonstrations as privileged information and identifies that models can rely on recurring reasoning-and-search output templates, producing trajectories that appear diverse but are largely agnostic to the input question, making the KL-based self-distillation signal uninformative.
Fan Yang, Rui Meng, Yuxin Wen· arXiv.org· 1 citation· ⚡1
The role of a frozen off-the-shelf instruct model as the teacher in on-policy distillation is investigated, and a key insight is revealed: the teacher reshapes the student's policy distribution so that subsequent RL converges to a superior solution that RL alone cannot reach.
Qi Ye, Zhi-Yuan Gu, Jingjie Xia et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
