Computing-And-Power Cooperative Scheduling for Distributed AI Training at Scale

Abstract

As AI training moves toward geo-distributed elastic infrastructures, its bottlenecks extend beyond GPU availability to variable energy and network conditions. The key challenge is translating heterogeneous placement, scaling, congestion, and training-stage effects into useful progress. We propose Effective Compute Utility (ECU), an interpretable progress signal, and build ECU-Aware, an online scheduler that coordinates data-center placement, elastic GPU allocation, and execution timing under budget and deadline constraints. Using Lyapunov optimization, ECU-Aware balances operating cost against long-term progress. Experiments with real traces and a small-scale prototype show that ECU-Aware improves the cost-latency-SLA tradeoff. In a 500-job workload, it reduces cost by 48.9 percent relative to a static policy, lowers average JCT by 36.0 percent, and decreases deadline misses by 17.2 percentage points compared with practical baselines.