Skip to content

Response-Informed Value Estimation for Cost-Aware Multi-Turn LLM Routing

Oct 2026 · 0 citations · 33 references
Computer Science

TL;DR

RIVET jointly trains two value estimators over a shared state-model representation: a deployable Routing Value Estimator that scores candidate models from pre-call information, and a training-only Response-Informed Value Estimator that additionally observes the realized response.

Abstract

Multi-turn LLM routing selects a model at every turn before its response is available, yet task success is revealed only when the interaction ends, leaving each intermediate choice without a label of its own. Logged trajectories, however, retain the response each call produced, which exposes what each turn required and how well the chosen model served it, evidence that outcomes alone hide. We introduce RIVET (Response-Informed Value Estimation for Turn-Level Routing), which exploits this post-call evidence during training while keeping routing strictly pre-call at deployment. RIVET jointly trains two value estimators over a shared state-model representation: a deployable Routing Value Estimator that scores candidate models from pre-call information, and a training-only Response-Informed Value Estimator that additionally observes the realized response. Conditioning on the response turns response-induced return variation from residual noise into an observed covariate, and we show that this reduces the leading asymptotic estimation error of the shared representation. On the MINT benchmark, RIVET solves more held-out tasks than the strongest multi-turn baseline while using 74.7% less total call cost.

View source

Similar papers

Preprint Nov 2026

Pathwise-Budgeted LLM Routing: Online Allocation with K-Bit Tail-Cost Learning

Firms using large language models under fixed budgets must decide which model to call and whether to retry a failed task. Because a call's charge is revealed only after completion, one long response can exhaust the remaining budget and leave later tasks unserved. Our objective is to maximize the number of tasks solved...

Han-Zhang Qin, Sophie H. Yu, Yue-Wei Zhang et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Whose turn is it? Context-guided model routing for stateful agents

Model routing can reduce the cost of LLM applications by assigning simpler requests to cheaper models without sacrificing the quality of the response. However, most routers treat requests as independent. Stateful agents do not operate under this assumption: a user turn may require multiple model and tool calls, its dif...

Anikesh Parashar, Avik Dutta, Vashu Chauhan et al. · 0 citations
#artificial intelligence Preprint Nov 2026

Ensemble Routing for Prompt-Adaptive Cost-Aware LLM Inference

PISA, a prompt-adaptive, cost-aware single-model router, and PISA-E, its extension to ensemble routing, are introduced and the effectiveness of prompt-adaptive capability estimation for both single-model routing and selective ensembling is demonstrated.

Saul Urso, Emanuele Carlini, Patrizio Dazzi et al. · 0 citations
#artificial intelligence Preprint Oct 2026

ORION: An Outcome-Driven LLM Advisor for RL-Based Multi-Domain 6G Slice Placement

Deploying 6G network slices across administrative domains requires an online decision for each arriving request: which domain hosts each virtual network function, how traffic crosses domain boundaries, and whether the slice is admitted. This joint decision problem is NP-hard under continuous arrivals. Experiments on a...

Mahmoud S. Mayaleh, Fetia Bannour, Nour-El-Houda Yellas · 0 citations

FlexRouter: Learning Complementary Model Sets for Flexible LLM Routing

Existing Large Language Model (LLM) routing methods score LLMs independently to select top-$k$ models. However, this ignores model correlations and enforces a rigid computational budget. Consequently, routers often select redundant models that share failure modes, limiting the overall probability of success. To address...

Wang Wei, Harry Yang, Tiankai Yang et al. · 3 citations
Open access Oct 2026

Online LLM Inference Routing for Multi-Agent Systems under Long-Term Budget and Layered Privacy Constraints

The Cloud-Edge-End (CEE) collaborative architecture has become a key paradigm for deploying highly intelligent multi-agent systems with low latency and high energy efficiency. Existing studies on efficient task offloading in CEE, however, either overlook long-term financial budget constraints from commercial API to...

Shi-Jing Yuan, Zi-Hang Li, Yu-Ming Zhang · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.