Skip to content

Pretrain Once, Route Anywhere: Towards a Foundation Model for LLM Routing

Sep 2026 · arXiv.org · Vol abs/2609.37362 · 1 citation · 27 references
Computer Science

TL;DR

This work introduces RouteFM, which learns to characterize anonymous candidate models from behavioral context and infer their target-specific capabilities, rather than binding routing decisions to fixed model identities or a single environment, and shows transfer across changes in domains, modalities, candidate pools, and context budgets.

Abstract

Large language model (LLM) routing aims to assign each query to the most suitable model from a heterogeneous candidate pool, improving the quality--efficiency trade-off of LLM inference. Existing routers are typically learned through local fitting: a router is optimized for a particular query workload and candidate pool, and often requires additional supervision or retraining as the routing environment changes. We ask whether LLM routing can instead be approached from a foundation-model perspective, learning a reusable routing capability that generalizes across tasks, candidate models, and deployment conditions. To this end, we introduce RouteFM, which learns to characterize anonymous candidate models from behavioral context and infer their target-specific capabilities, rather than binding routing decisions to fixed model identities or a single environment. Through episodic pretraining across heterogeneous routing environments, this capability can be reused by a frozen router and adapted to new environments through context alone. Experiments demonstrate transfer across changes in domains, modalities, candidate pools, and context budgets, with the largest gains when behavioral evidence is limited. On MMR-Bench, which is excluded from pretraining, RouteFM outperforms the strongest baseline by 2.23 quality points with only eight observations per candidate. These results support moving LLM routing from repeated local fitting toward a pretrain once, route anywhere paradigm. Our code is publicly available at https://github.com/LAMDA-Model-Reuse/RouteFM.

View source

Similar papers

SeLMRoute: Probabilistic Semantic Evidence for Large Language Model Routing

SeLMRoute is introduced, a routing framework that separates the extraction of candidate-independent semantic evidence from the learning of candidate performance and the application of deployment objectives and achieves the highest mean performance among the evaluated semantic, dense, lexical, and domain-level represent...

Vasilis Perifanis, Nikolaos Pavlidis, Symeon Symeonidis · 1 citation

FlexRouter: Learning Complementary Model Sets for Flexible LLM Routing

Existing Large Language Model (LLM) routing methods score LLMs independently to select top-$k$ models. However, this ignores model correlations and enforces a rigid computational budget. Consequently, routers often select redundant models that share failure modes, limiting the overall probability of success. To address...

Wang Wei, Harry Yang, Tiankai Yang et al. · 3 citations
#natural language process... Preprint Oct 2026

Breaking the Tie: A Cluster-Aware Routing Framework for Large Language Models

With the rapid development of artificial intelligence, the emergence of various Large Language Models (LLMs) has created a rich model ecosystem. However, this also brings a key challenge: how to select the optimal model for a specific user query. LLM routing addresses this need by dynamically assigning queries to the m...

Yao Lu, Zhai-Yuan Ji, Ya-Xin Gao et al. · 0 citations
Preprint Aug 2026

RLCascadeRouter: Quality-Estimator-Free Cascade Routing via Reinforcement Learning

RLCascadeRouter is a quality-estimator-free framework that formulates cascade routing as a Markov decision process with actions comprising ``stop''and model selection, and uses trajectory returns and advantages to directly optimize the performance-cost objective.

Shihong Huang, Sheng-Jie Wang, Hong-Yao Ma et al. · 1 citation

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.