Skill-routed Evaluation with Evolvable Knowledge (SEEK) is proposed, which externalizes specific search evaluation criteria into a skill bank, dynamically routes relevant skills for each query-result list pair, and employs a task-adapted listwise evaluator to produce page-level judgments and failure mode attribution.
Abstract
Search quality evaluation provides essential supervision and diagnostic signals for the development and iteration of industrial search systems. Although large language models (LLMs) offer a scalable alternative to manual assessment, reliable automatic evaluation remains challenging: users experience search results at the page level, while the applicable evaluation criteria are multi-dimensional and continuously evolving. Packing all evaluation criteria into a unified prompt introduces irrelevant context and potential criterion interference, whereas internalizing them through post-training tightly couples rule updates with costly model retraining cycles. To address these issues, we propose Skill-routed Evaluation with Evolvable Knowledge (SEEK). Specifically, SEEK externalizes specific search evaluation criteria into a skill bank, dynamically routes relevant skills for each query-result list pair, and employs a task-adapted listwise evaluator to produce page-level judgments and failure mode attribution. A two-stage training pipeline teaches the evaluator to align evaluation criteria with human preferences, while a replay-gated skill bank allows recurring evaluation knowledge gaps to be incorporated without model retraining. Experiments on industrial short-video search show that SEEK improves listwise quality evaluation accuracy and achieves significant progress in attribution diagnosis. SEEK has been deployed at Kuaishou, a short-video platform with over 400 million daily active users, significantly improving the scale and quality of online search evaluation.
SPARO (Skill, Prompt, And Routing Optimization), a framework that jointly optimizes task instructions, reusable skill blocks, and routing rules, is proposed, suggesting that effective language-program optimization depends not only on discovering useful task knowledge, but also on deciding where that knowledge should be...
Hao-Ran Shou, Hao-Yue Liu, Yun-Chen Huo et al.· arXiv.org· 0 citations
StraTune (strategy-guided skill tuning), which lets a frozen optimizer LLM choose the revision operator at every round from the optimization state, which is defined as the current execution feedback together with the recorded outcomes of earlier strategies and forms.
Ze-Ping Liu, Yan Li, Ni Lao et al.· arXiv.org· 0 citations
Deploying frozen learned skills, such as Vision-Language-Action (VLA) policies, in new environments requires identifying initiation configurations that support reliable execution. In skill composition, an initiation configuration affects not only the current skill but also the physical state passed to subsequent skills...
Results show that persistent strategy experience provides effective guidance for automated fine-tuning on unseen tasks, and propose Strategy Accumulation and Guided Execution (SAGE), a two-stage framework that makes automated fine-tuning search cumulative.
Haoran Zhao, Wei Du, Dingwen Yang et al.· 0 citations
In the construction sector, technical bid evaluation is crucial for maintaining equity, evaluating contractor capabilities, and allocating resources efficiently. However, traditional methods rely heavily on manual expert review, which can introduce subjective biases and efficiency bottlenecks. While the existing litera...
A concrete architecture for cumulative executable search and a versioned account of its cross-modal development outcomes are added, including a concrete architecture for cumulative executable search for cumulative executable search for EES.