Large language model (LLM) training today runs on clusters spanning thousands of GPUs. While this scale enables rapid model advances, developing, debugging, and performance-tuning, the training framework inevitably becomes complex and costly. This is because engineers often need to reproduce production behaviors to dia...
Shao-Ke Xi, ChonLam Lao, Bo-Yi Jia et al.· Proceedings of the ACM SIGOP...· 0 citations
This work argues for co-design, that facilities should treat agents as first-class principals where agents become facility-aware tenants, and outlines the resulting challenges and opportunities in compute, storage, agent memory, and safety.
Yun-Jia Zheng, Bintang Dwi Marthen, Zachary Pan et al.· 0 citations
This work presents Meta's network architecture and software stack designed to support one of the world's largest RoCE fabrics, currently connecting over 100,000 GPUs across multiple datacenter buildings, and introduces a scalable initialization strategy that reduces startup times by 11× via eager process group creation...
Hongyi Zeng, Min Si, Pavan Balaji et al.· Conference on Applications,...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
