Skip to content

Embedding Geometry Shapes LLM Routing: Tracing Failure Modes and Improving Generalization

Oct 2026 · 0 citations · 44 references
Computer Science Mathematics

TL;DR

The geometry of semantic embeddings is studied by studying the geometry of semantic embeddings to trace a clear failure mode: features that determine the neighbors of unseen prompts are an unreliable proxy for LLM selection.

Abstract

With the growing number of LLMs available, selecting which one to use for a prompt has become a key question. A large part of the routing literature relies on semantically embedding prompts into a vector space and then applying a lightweight classifier to the embedded representations. These methods perform robustly in-distribution but they struggle to generalize to unseen tasks. By studying the geometry of semantic embeddings, we trace a clear failure mode: features that determine the neighbors of unseen prompts are an unreliable proxy for LLM selection. We therefore replace the semantic embedding with the following map: we run a partial forward pass of Qwen2.5-7B-Instruct on the prompt and use the hidden state of its last token at a deep layer as the representation. When applying kNN and other simple classifiers to this new representation, we obtain a clear result in generalization: on average across cost penalties, their performance improves. Moreover, kNN and another simple classifier that we propose outperform every evaluated semantic-embedding baseline. Our two simple methods also beat a recent SoTA hidden-states based router on RouterBench.

View source

Similar papers

Routing Without Embeddings: Fast And Interpretable Routing With Regular Expressions

RegEXROUTE is introduced, a pipeline that uses sparse autoencoders (SAEs) to discover interpretable regular-expression (regex) features and establishes explicit, interpretable text features as a practical basis for designing and understanding LLM routers.

Yi-Fan Lu, Qi-Yue Zhang, Haotian Shan et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Compute Has Shape: Self-Morphing the Internal Computation of Frozen LLMs

Test-time scaling improves large language models through additional computation, but how to adapt the topology of internal computation to individual inputs remains underexplored. Existing approaches largely fix recurrent or parallel architectures during training, or reconfigure frozen models along a single structural a...

Jun-Xi Wu, Yi-Wen Guo, Shu-Tao Xia · 0 citations
#artificial intelligence Preprint Oct 2026

From the Zoo to the BULLPEN: Sample-Efficient LLM Profiling via Multi-Channel Model Embeddings

BULLPEN (Behavioral Understanding of Large Language models via Profile ENcodings), a framework that extends EmbedLLM in three ways and is the first framework to read a model's own answer text jointly with its correctness and the question text, is introduced.

Russell Ro, Qi-Ming Cui, Reza Abbasi-Asl · 0 citations
Preprint Nov 2026

SPARSEUP: Towards Better Recipes for Training Modern Learned Sparse Retrievers

Learned sparse embeddings are a powerful alternative to dense retrieval, yet they have fallen behind single-vector and late interaction models. One reason for their limited adoption is that they are notoriously difficult to train, generally requiring very tight conditions to converge. We present a recipe for training S...

Thibault Formal, Pierre Erbacher, Cosimo Rulli · 0 citations
#machine learning Preprint Oct 2026

Contrastive Linearized Laplace: Loss Geometry, Valid Repairs, and Reliable Cores

While contrastive learning is widely used to learn deterministic representations, it does not inherently provide estimates of uncertainty. Post-hoc uncertainty estimation methods, such as the Linearized Laplace approximation (LLA), offer a way to quantify uncertainty without retraining the encoder. However, we show tha...

F. J. Sáez-Maldonado, Luis A. Ortega, Andrés R. Masegosa et al. · 0 citations

Block-Sparse Attention with Semantic-Geometric Decoupled Routing

Semantic-Geometric Decoupled Routing is proposed, a training-free block routing framework that shifts semantic aggregation to the pre-RoPE space and reconstructs geometric bias with an offline structural prior and relative block distances and yields an explicit closed-form block routing score without token-level search...

Xin-Wei Long, Wei-Gao Sun, Weibo Gao et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.