Yu-Hao Meng

PipeComm: Maximizing Link Utilization Through Pipeline-Aware Collective Communication Synthesis

Efficient collective communication is crucial for distributed training. While recent topology-aware synthesis approaches attempt to optimize communication based on the network topology, they struggle with heterogeneous environments where links exhibit distinct bandwidths and asymmetric connections. The main inefficienc...