Who Pays for Efficient Inference? Confidence-Based Small-to-Large LLM Routing Across Tasks and Languages
A systematic evaluation of confidence-based routing across three model families, four languages, and three task types, covering 72 model, dataset, language, and scale configurations and 42,288 predictions shows that confidence routing is task-dependent and can allocate computation unevenly across languages.
Md Sabbir Hossen, Anichur Rahman, Pabon Shaha et al.
· 0 citations