LLM routers pick a cheap or expensive model per request by its content, and many gateways and some cloud platforms can log that choice with content logging off. We measure this privacy channel beyond token counts, accounting for noisy labels and repeated prompts. We run pre-registered studies on 1.7 million real reques...
After a weak verifier accepts a large-language-model response, a second call may resample or reroute, but current evidence does not identify when to resample rather than reroute, so stopping debt exists, but current evidence does not identify when to resample rather than reroute.
It is found that every one of five epilogue faults -- scale precision, double rounding, multiplication order, output truncation, fused ordering -- moves the output by at most a single bfloat16 spacing, and by exactly one whenever it moves it at all, across 5,880 cells.
An online resample-or-reroute (RoR) allocation policy driven by estimated marginal correctness per unit cost is proposed, and its behavior is grounded in the recoverability asymmetry between selection and sampling.
Two GPU kernels implementing the same scaled INT8 GEMM interface are usually treated as interchangeable but cross-implementation FP8 GEMM shows a different signature: both the prevalence and the magnitude of differences grow with reduction depth, while the INT8 fraction stays at parts per million and within one spacing...
Teng-Ruei Chen· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.