Open access
May 2025
ZenFlow: Enabling Stall-Free Offloading for LLM Training
ZenFlow is presented, a new offloaded training framework that decouples updates between GPU and CPU by prioritizing important gradients and achieves up to 5× end-to-end speedup, 2× lower PCIe traffic, 85% less GPU stalls, and up to 72% lower cloud GPU cost, all while preserving accuracy.
Ting-Feng Lan, Yu-Sen Wu, Bin Ma et al.
· Proceedings of the ACM on Ma... · 8 citations
· ⚡3