Aug 2026· Proceedings of the ACM SIGCOMM 2026 Conference· 0 citations· 82 references
Abstract
Current GPU Collective Communication Libraries (CCLs) employ predefined schedules optimized for stable environments. Their supported schedules and selection logic are fixed at communicator initialization, which fails to account for evolving runtime conditions, such as workload characteristics and hardware health status. Consequently, long-running GPU jobs experience suboptimal performance after hours or days of execution, which translates into longer job completion times and wasted GPU cluster resources. To address this problem, we present Theseus, a novel CCL backend that provides schedule-level runtime adaptivity. It admits user-defined schedules and selection policies. As runtime conditions change, Theseus selects suitable schedules using cluster-wide runtime attributes beyond CCL-internal metrics. Moreover, it hot-swaps from the previous schedule consistently across GPUs with low overhead. Theseus acts as a drop-in replacement to facilitate integration. We evaluate Theseus extensively on various GPU workloads with intuitive policies. Compared with NCCL, Theseus achieves up to 1.61X speedup of communication time in stable environments and 2.46X in dynamic environments. It improves end-to-end job completion time by up to 1.84X while incurring comparable or lower overhead.
Understanding the impact of hardware configuration, infrastructure investments, and operational policies on job wait times in high-performance computing (HPC) systems is a challenging problem primarily due to the lack of effective tools that are built on controlled, real-world observations across different system configurations. In this work, we present a data-driven framework that leverages historical job traces to estimate the impact of resource modifications on queue performance. Using job data collected from multiple HPC systems during periods with no hardware changes, we construct a synthetic scheduler that replays real job submission patterns and resource requests, while using ground-truth runtimes to model execution. This enables controlled, counterfactual evaluation of infrastructure changes without modifying production systems. We perform experiments by scaling system resources, including GPUs and CPU cores. Our results on GPU-based systems show that increasing GPU capacity leads to significant reductions in wait times, while CPU-only scaling provides minimal benefit. To quantify these effects, we introduce the Weighted Wait-Time Score (WWS), a bounded metric that captures both typical and tail wait-time behavior. Our experiments show that informed resource scaling improves WWS (up to 92% relative gain in WWS for low-baseline systems), capturing gains in both typical and tail wait-time behavior under this metric. We further formulate a cost-aware optimization framework to guide resource allocation under budget constraints. Our approach offers a data-driven way to evaluate HPC upgrades and supports future predictive optimization and better resource use. Overall, this work provides (i) a trace-driven framework for evaluating resource scaling effects, (ii) a bounded metric (WWS) for comparing wait-time performance, and (iii) a pathway to optimal user wait time optimization based resource allocation in HPC systems.
Bipin Gaikwad, Shraddha Singh, M. Joshi et al.· Practice and Experience in A...· 0 citations
Training large language models on limited hardware is increasingly a scheduling problem across GPU compute, host memory, PCIe transfer, and storage bandwidth. Existing offloading systems reduce GPU residency, and MegaTrain shows that a CPU-master layer-streaming executor can train large models on a single GPU, but fixed checkpointing and placement heuristics still leave communication exposed on the critical path. We propose LazyTrain, an optimization layer over a layer-streaming executor. LazyTrain formulates checkpoint selection, activation placement, recomputation, and CPU-GPU-NVMe communication overlap as a mixed-integer scheduling problem, then executes the solved policy during training. It further couples 8-bit optimizer states with fast gradient clipping as a single Hybrid 8-bit operator: state compression reduces optimizer-state memory, while fast clipping counteracts the additional CPU-side update overhead. Across H800 experiments from Qwen2.5-3B to Qwen3.6-27B, LazyTrain improves sustained TFLOPS over matched baselines runs by approximately 1.24$\times$; RTX 3090 experiments likewise increase the maximum feasible batch size by one at each model scale. In the primary Qwen3.6-27B H800 MetaMathQA run, LazyTrain reaches 219.95 TFLOPS and 1361 tokens/s at batch size 72, peaks at 68.84\,GB of GPU memory, and obtains 95.42\% exact-match accuracy on the full evaluation split. The source code is available at https://github.com/DataArcTech/LazyTrain.
Xiaojun Wu, Cehao Yang, Honghao Liu et al.· 0 citations
ElastiCo is presented, an elastic co-location framework that enables training and inference workloads to safely share GPUs through three integrated mechanisms that decomposes the resulting multi-resource allocation problem into per-job configuration selection subproblems via dynamic per-resource shadow prices.
Jinghao Wang, Yihang Zhou, Xiaoyang Sun et al.· 0 citations
We introduce PTXBench, a benchmark for evaluating and adapting large language models (LLMs) to use architecture-specific PTX for GPU kernel optimization. PTXBench measures functional correctness, whether selected target instructions execute at runtime, and speedup over frontier libraries across GEMM and attention workloads on H100 and B200 GPUs. Our evaluation shows that architecture-specific PTX capability remains uneven: success rates fall substantially on complex attention backward workloads, and executing the target instructions does not necessarily translate into competitive performance. No evaluated model consistently matches frontier libraries across the suite. We further adapt Qwen3.6-27B using supervised fine-tuning. Repair-conditioned training improves several tasks, but generalization remains uneven; data coverage, balance, and the quality of the reasoning teacher matter in addition to dataset size. PTXBench provides an auditable testbed for measuring and improving LLMs'ability to exploit evolving GPU architectures.
Genghan Zhang, Yixin Dong, Chengze Fan et al.· 0 citations
Festina is presented, a profiling-guided, power-aware control plane to minimize cluster-wide energy for serverless LLM serving and performs energy-aware workload consolidation to reduce GPUs'static power consumption via SLO-aware migration.
Tianyu Wang, Gourav Rattihalli, A. Dhakal et al.· arXiv.org· 0 citations
AI workloads increasingly demand high parallelism and efficient on-chip memory utilization on modern GPUs. However, the limited capacity of shared memory often constrains thread-level parallelism, while portions of the L2 cache remain underutilized. In this work, we propose SAI, a mechanism that virtualizes shared memory into the L2 cache to improve GPU performance for AI applications. SAI dynamically activates virtualization based on runtime resource usage, enabling additional CTAs with minimal architectural modifications. We further introduce an L2 cache management strategy that integrates associativity-based virtual page allocation and a replacement information table, reducing page-swapping overhead while preserving L2 cache performance. Experimental results demonstrate that SAI achieves an 18.6% performance improvement over the baseline design and outperforms SMILE-opt, our reproduced state-of-the-art baseline, by 8.8%. Moreover, SAI captures 60.2% of the performance gain delivered by the idealized Double SMEM design while reducing energy consumption to 87.7% of the baseline. These results highlight the effectiveness of SAI in enhancing thread-level parallelism and optimizing on-chip memory for AI workloads.
Hanqing Li, Tiejun Li, Sheng Ma et al.· ACM Transactions on Design A...· 0 citations