Dynamic LLM routers promise to cut inference costs by sending each query to the cheapest model that can answer it correctly. We analyze six commercial routers across 14 settings on a diverse benchmark spanning eight task categories, finding that none of them outperforms a router that randomly selects between two well-c...
Sam Wang, Julia White, Sahibzada Allahyar et al.· 1 citation· ⚡1
This work presents ToolGate, which turns repeated answer checking and difficulty screening into an auditable process while leaving domain design and final review to experts.
Ke Zhang, Yan-Kang Liu, Roya Zandi et al.· 0 citations
It is argued that evaluations of scientific agents should report not only accuracy, but also item-level retention, output-access sensitivity, trajectory failures, and where the computation chain breaks.
Ke Zhang, Sahchit Chundur, M. J. Qomi et al.· 0 citations
This work uses a full $2^3$ factorial design to decompose three recurring interventions in formalization pipelines: parametric expert drafting, Mathlib/context search, and Lean elaboration feedback, suggesting that formal validity, proof-oriented Lean competence, and faithful statement generation should be reported sep...
Ke Zhang, P. Gallardo, S. Murthy et al.· arXiv.org· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.