X-Router: Decoupling Knowledge and Reasoning for Cost-Effective LLM Inference

Abstract

Large Language Models (LLMs) are often augmented with Retrieval-Augmented Generation (RAG) and Chain-of-Thought (CoT) prompting, yet static “always-on” use is computationally wasteful. Existing adaptive meth-ods typically optimize a single axis, overlooking that evidence need and reasoning depth are only partially correlated. We present X-R OUTER , a dual-axis routing framework that separates retrieval necessity from reasoning necessity under a user-defined cost–quality trade-off. Offline, X-R OUTER profiles four pipelines (D IRECT , RAG, C O T, RAG+C O T) and derives supervision by selecting the utility-maximizing strategy that trades answer quality against token usage and latency. On-line, a compact dual-head router, conditioned on cost weights, uses lightweight probes— retrieval-score dispersion (NQC) and single-pass draft negative log-likelihood (NLL)—to decide whether to invoke RAG and/or CoT without sampling or model internals. Across six QA benchmarks, X-R OUTER reduces