Tuning Collective Patterns to Alleviate Congestion in Shared AI Clusters
This paper builds a system, REACT, that tunes the recurring pattern of data exchange between GPU nodes (known as communication collectives) in response to congestion, and improves communication performance by 13%-38% under network congestion.
Eashan Gupta, Yong-Zhou Chen, Apoorve Mohan et al.
· 0 citations