A small number of GPU kernels often determines the efficiency of large language model training and inference. High-performance kernel development typically relies on expert knowledge through manual implementation, collaboration with general-purpose models, or specialized agents equipped with domain knowledge, evaluatio...
Yao-Jian Chen, Da-Zhi Peng, Zhui Zhu et al.· 0 citations
Modern processors increasingly provide matrix engines whose peak arithmetic throughput greatly exceeds conventional SIMD, but scientific applications rarely realize this advantage end to end. We examine this gap in SPECFEM3D's dominant stiffness operator on the Arm LX2 CPUs that power the flagship Lineshine supercomput...
Yinuo Wang, Lin Gan, Tianqi Mao et al.· 0 citations
StateFlow is presented, a sequence pipeline parallelism system for models with linear recurrence that partitions each sequence into chunks and schedules their execution while propagating boundary states and gradients across chunks, thereby reducing activation lifetimes and improving training throughput.
Wenxuan Zhao, Yingfa Chen, Xu Han et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.