Large language models (LLMs) now serve as the backbone of agents that reason over many turns, call tools, and pursue long-horizon tasks. To avoid recomputation, a model stores the attention keys and values of all past tokens in a key-value (KV) cache. Each layer maintains its own cache, so KV-cache memory can become th...
This analysis supports a correctness-prediction bottleneck hypothesis: current routers primarily learn global-average model performance trends rather than fine-grained, query-specific routing signals, and collectively fail on queries that require instance-specific routing decisions.
Yi-Fan Lu, Qi-Yue Zhang, Shen-Run Zhang et al.· 0 citations
RegEXROUTE is introduced, a pipeline that uses sparse autoencoders (SAEs) to discover interpretable regular-expression (regex) features and establishes explicit, interpretable text features as a practical basis for designing and understanding LLM routers.
This work introduces a controlled and MAS-demanding diagnostic benchmark for representative MAS efficiency methods and shows that many reported gains are setup-dependent and may arise from structural collapse, disabled tool pathways, or starting systems where random pruning already preserves accuracy, rather than robus...
Jia-Mu Zhang, Ling-Xi Zhang, Peng-Jun Lu et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.