OpDisPath: Optimal Communication Scheduling in GPU Clusters Through Disjoint Path Selection
It is proved that for non-oversubscribed networks, there exists an optimal communication scheduling scheme capable of completely avoiding bandwidth contention among the scheduled flows in each collective-communication stage and achieves up to 77.5% reduction in iteration time compared to state-of-the-art methods.