This work argues that a general workload representation, a verifiable mutation space, and an implementation-independent evaluator are required to enable the AI-driven LLM inference system architecting loop, and presents the RoofLang domain-specific language (DSL) that provides these features.
Ziyue Yang, Yu-Ting Jiang, L. Qu et al.· 0 citations
OptiFlow is presented, among the first LLM-driven frameworks for automated design of high-performance collective communication algorithms, with key insight is a two-layer decomposition: the LLM generates compact data-movement intent expressed in a domain-specific language, while deterministic scheduling algorithms comp...
Fei Long, Ziyue Yang, Kaihui Gao et al.· Asia-Pacific Workshop on Net...· 1 citation
From a request's prefill expert activations, ELDR builds an expert signature predicting the experts it will activate during generation, which reduces median TPOT by 5.9-13.9% over the strongest of four load-balancing baselines across three MoE models and two workloads.