Pathwise-Budgeted LLM Routing: Online Allocation with K-Bit Tail-Cost Learning
Firms using large language models under fixed budgets must decide which model to call and whether to retry a failed task. Because a call's charge is revealed only after completion, one long response can exhaust the remaining budget and leave later tasks unserved. Our objective is to maximize the number of tasks solved...