Large language models (LLMs) now show strong reasoning and coding abilities, raising the prospect that they could assist in developing and optimizing the very infrastructure that powers them. Existing benchmarks, however, mainly focus on isolated kernels, predefined operators, or pre-specified optimization targets, and...
This work presents $\Phi$-Bench, a benchmark for systematically evaluating LLMs on engineering the LLM infrastructure stack and spans tasks of varying complexity, ranging from localized kernel-level function completion to long-horizon implementation and end-to-end system optimization.
Lei-Lei Ding, Shu-Min Wang, Yu-Ting Huang et al.· 1 citation
A larger-batch configuration reduces time-to-target only when its throughput gain exceeds its samples-to-target penalty, and a larger-batch configuration reduces time-to-target only when its throughput gain exceeds its samples-to-target penalty.
Zi-Niu Li, Jinbo Wang, Guan-Hua Huang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.