This work presents a trace-driven study of network sensitivity for training and inference workloads at scales up to 4,096 GPUs, and combines signature-based trace decomposition with linear programming to make broad network-parameter sweeps feasible.
Abstract
AI workloads on supercomputers make interconnect performance a primary design constraint. Hardware specifications and microbenchmarks alone do not show how real workloads respond to latency, bandwidth, and jitter. We present a trace-driven study of network sensitivity for training and inference workloads at scales up to 4,096 GPUs. Our methodology combines signature-based trace decomposition with linear programming to make broad network-parameter sweeps feasible. It preserves the recorded NCCL schedules and program dependencies, and estimates a lower bound on runtime under the LogGP model. We validate the compositional analysis against monolithic linear programs, LogGOPSim, and hardware measurements at feasible scales. The resulting latency-bandwidth envelopes identify low-sensitivity regions and performance cliffs that vary substantially across workloads. They also show when bandwidth upgrades provide meaningful returns and when additional network performance has little effect. These results translate execution traces into practical guidance for evaluating network configurations while making the model assumptions explicit.
AI compute is becoming large and fast-changing electrical loads. Its impacts on data center infrastructure and power grids depend not only on total energy consumption, but also on how their power draw moves over sub-second to minute timescales, including peaks, ramps, and synchronized fluctuations across accelerators....
Jian-Sheng Wang, J. Singh, Hao Xu et al.· 0 citations
In this work, we perform an in-depth characterization of the microarchitectural performance of modern datacenter workloads. Google recently released hundreds of gigabytes of trace data from production, offering a unique view into current-day warehouse-scale workload behavior. Through simulating 359 cores spanning 12 wo...
Matthew Giordano, Akanksha Jain, E. Deiana et al.· IEEE International Symposium...· 1 citation
LMTracer is presented, a fine-grained and real-time performance profiling framework for production LLM services that embed profiling logic into the execution through graph-embedded probing and streaming buffered profiling data to CPUs on demand to keep the execution of user kernels uninterrupted.
Wei Liu, Yong-Chao He, Bo-Han Zhao et al.· Proceedings of the ACM SIGOP...· 0 citations
Modern inference engines have significantly advanced the scalability of large models through innovations in serving techniques, proving instrumental in driving the widespread adoption of AI. However their true potential is constrained by traditional server architecture where reliance on host CPUs and operating systems...
Shaunak Galvankar· Proceedings of the 35th ACM...· 0 citations
Near Data Processing (NDP) has the potential to significantly improve system performance and energy by alleviating data movement bottlenecks. However, most NDP proposals pose heavy requirements for the software stack, data layout, and/or the underlying hardware. To broaden NDP adoption, this work focuses on a modular h...
Shreyas Singh, Pratyush Nandi, Lin Jia et al.· arXiv.org· 0 citations
StreamTrace is presented, a streaming trace analysis system that enables efficient analysis of massive traces on a single computing node with bounded memory consumption and introduces two key techniques: communication pattern-aware chunk partitioning that minimizes cross-chunk dependence and dynamic priority-based chun...
Yu-Yang Jin, Ji-Dong Zhai· International Conference on...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.