Jul 2026· International Conference on Supercomputing· 0 citations· 52 references
Computer Science
TL;DR
DEFT is presented, an energy-aware scheduling framework that jointly optimizes task-to-device assignment and per-GPU DVFS configuration for task-based multi-GPU applications and employs a cost-model–driven strategy that integrates slack awareness, throughput awareness, and explicit modeling of task execution cost, inter-GPU data movement, and DVFS transition overheads, enabling coordinated placement and frequency decisions under dynamic runtime conditions.
Abstract
Energy efficiency has become a first-order concern in modern high-performance computing systems, as it directly determines achievable throughput under fixed power budgets. Although Dynamic Voltage and Frequency Scaling (DVFS) provides an effective mechanism for reducing GPU energy consumption, existing runtime systems decouple DVFS from task placement and inter-GPU communication, focus on single-GPU execution, or cannot adapt frequency to task granularity and runtime contention in multi-GPU environments. Consequently, current schedulers fail to capture the tight coupling between task placement, frequency selection, and inter-GPU data movement that fundamentally governs energy–performance trade-offs on multi-GPU systems. This paper presents DEFT, an energy-aware scheduling framework that jointly optimizes task-to-device assignment and per-GPU DVFS configuration for task-based multi-GPU applications. DEFT employs a cost-model–driven strategy that integrates slack awareness, throughput awareness, and explicit modeling of task execution cost, inter-GPU data movement, and DVFS transition overheads, enabling coordinated placement and frequency decisions at task granularity under dynamic runtime conditions. We prototype DEFT within the CUDASTF runtime and demonstrate its effectiveness across five optimization objectives. The evaluation shows that DEFT reduces energy consumption by 14.8% and 4.8% on average on NVIDIA L40S and L4, and reduces EDP by 9.9% and 3.7%, respectively, while maintaining performance within 1.5% of the fastest baseline.
Festina is presented, a profiling-guided, power-aware control plane to minimize cluster-wide energy for serverless LLM serving and performs energy-aware workload consolidation to reduce GPUs'static power consumption via SLO-aware migration.
Tianyu Wang, Gourav Rattihalli, A. Dhakal et al.· arXiv.org· 0 citations
Multi-chip-module (MCM) GPUs improve scalability by integrating multiple GPU chiplets, but spatial multitasking introduces severe performance interference due to non-uniform memory access and asymmetric inter-chip bandwidth. Existing GPU multitasking schedulers rely on runtime profiling metrics to guide placement decisions. These reactive approaches observe the consequence of interference rather than its structural cause, vary across inputs and co-running workloads, and incur non-trivial overhead. This paper presents SA-Scheduler, a structure-derived bottleneck-aware scheduling framework for multitasking MCM-GPUs. Instead of profiling execution behavior, SA-Scheduler derives hardware bottleneck regimes directly from kernel structure prior to execution. Through offline architecture-aware modeling, we extract iteration-normalized features such as warp-level working-set footprint and global memory intensity (GMI). A fine-tuned lightweight code model (Qwen-2.5-Coder-1.5B) assists semantic structural extraction for CUDA kernels. These features are translated via an analytical bottleneck model into distinct interference regimes (C, M1, M2, and M3), characterizing how kernels interact with hierarchical and asymmetric memory resources under co-execution. Based on the inferred regimes, SA-Scheduler determines chip placement without hardware modification or runtime bottleneck profiling. For streaming multiprocessor (SM) allocation and optional power-gating decisions, SA-Scheduler further employs lightweight counter-based feedback to tune resource usage under bounded overhead. Across 105 two-program workloads, SA-Scheduler improves system throughput (STP) by 17.63% and 10.48% over intra-chip and inter-chip scheduling, respectively, while increasing fairness by up to 33.27%. An energy-aware Power Mode further reduces average power by up to 25.39% under bounded throughput degradation. These results demonstrate that structure-derived bottleneck inference provides a principled and scalable foundation for multitasking on future MCM-GPUs.
Tiejian Zhang, Guangda Zhang, Lu Wang et al.· 0 citations
GPU energy consumption represents a primary operational cost for Large Language Model (LLM) inference services. Despite the inherent variability of production workloads, most deployments rely on static power configurations that fail to exploit the non-linear relationship between power limits and performance. Consequently, opportunities to reduce energy overhead while maintaining Service Level Agreements (SLAs) are often missed. While modern GPUs provide hardware-level powercapping interfaces, their systematic application in a feedbackdriven manner to satisfy stringent latency guarantees remains significantly underexplored. This paper proposes a lightweight, non-intrusive adaptive GPU power-capping mechanism based on a PID controller that dynamically adjusts power limits using SLA feedback. The controller operates directly through vendor power-limit interfaces and incorporates guardrails such as deadbands, rate limiting, utilization-aware gating, and antiwindup protection. We evaluate the approach on a server with eight NVIDIA H200 GPUs running the qwen3:32b model under a time-varying Poisson workload, with SLA compliance defined over TTFT. Results show that static caps expose a rigid energy-SLA trade-off, whereas the adaptive controller achieves a 3.84% reduction in total energy consumption relative to a 700 W baseline while maintaining empirical SLA non-inferiority. These findings demonstrate that feedback-driven GPU power modulation can improve energy efficiency without compromising latency stability.
Alex F. R. Trajano, C. Costa, Francisco V. J. Nobre et al.· Annual International Compute...· 0 citations
The massive energy consumption of GPU-accelerated AI workloads challenges sustainable computing. We observe that execution asynchrony (e.g., CPU-GPU, concurrent streams, multi-GPU) creates slack, allowing non-critical kernels to run at lower frequencies to save energy without impacting end-to-end latency. However, existing approaches fail to simultaneously achieve workload generality and fine-grained slack discovery, while high-fidelity modeling incurs prohibitive overhead. We present EMO, a lightweight framework exploiting these fine-grained opportunities. First, to identify where to optimize, EMO constructs a low-level dependency graph capturing asynchrony and performs what-if timing analysis to precisely identify slack windows. Second, to determine how to optimize, EMO introduces dependency-aware kernel packing. It aggregates kernels to preserve critical paths while collapsing redundant details, enabling high-fidelity latency-energy modeling with minimal profiling cost. Finally, EMO combines graph analysis and pack-level models to formulate energy optimization as a constrained combinatorial problem, efficiently solving for optimal frequency policies under given latency targets. Evaluations show EMO reduces energy consumption by 15%--28% with only 2%--5% performance loss and negligible overhead.
Jiyu Luo, Shaoyu Chen, Jingwei Sun et al.· 0 citations
Energy consumption is a key limitation in high-performance computing on heterogeneous CPU-GPU systems. This work studies how hardware configuration affects energy-to-solution under realistic workloads. We study energy efficiency regimes using molecular dynamics benchmarks (GROMACS and AMBER) and a stress-test benchmark (FIRESTARTER) on systems with A40, A100, H100, and H200 GPUs and Intel Ice Lake CPU, varying frequency scaling and power cap. We show that energy-to-solution exhibits workload- and architecture-dependent transitions between efficient and inefficient regimes, driven by nonlinear GPU power-frequency scaling. We introduce an interpretable analytical model that decomposes GPU power into linear and nonlinear components, identifying a workload- and architecture-dependent transition frequency beyond which efficiency degrades. The model fits empirical data with low error and highlights the role of baseline power, nonlinear power behavior, and transition frequency as the dominant parameters governing energy efficiency. Power capping is generally less effective for efficiency tuning than frequency reduction, especially for workloads that operate far from thermal design power. Overall, energy-efficient HPC execution is a configuration-dependent problem with identifiable regime shifts, and we provide model-driven guidance for selecting operating points.
Ayesha Afzal, Markus Manfred Li, Michael Panzlaff· 0 citations
CPU–GPU heterogeneous computing platforms are increasingly ubiquitous in mobile embedded systems, including smartphones, tablets, and unmanned aerial vehicles, due to their superior computational throughput. However, given the strict energy limitations inherent to battery‐powered devices, the development of power‐efficient, real‐time computing architectures remains a critical design challenge. This paper proposes a power‐efficient scheduling methodology leveraging thread‐based parallel programming to optimize resource utilization across CPU–GPU architectures using the taskflow framework. By explicitly modeling thread‐level runtime behavior across heterogeneous computation units, the scheduling objective is formulated as an integer nonlinear programming problem aimed at minimizing total system power consumption. A state‐transition framework is derived, and a dynamic programming algorithm is introduced to optimize computing task assignment and computational resource allocation. Extensive empirical evaluations demonstrate that the proposed dynamic programming strategy significantly accelerates convergence and yields superior power‐efficient schedules compared to baseline methodologies, including particle swarm optimization. Furthermore, validation on a physical platform confirms the practical efficacy and accuracy of the proposed scheduling architecture.
Xulong Wu, Biao Hu· Concurrency and Computation· 0 citations