TL;DR
A mathematical model is proposed to predict throughput and energy consumption for concurrently executing CV workloads on edge GPU accelerators and can be integrated into functional simulation frameworks for edge–cloud deployment studies.
These findings demonstrate that contemporary generative workloads can be effectively supported by decentralized edge infrastructures, providing practical insights for the design of energy-efficient and heterogeneous local AI systems.
This work evaluates whether Model FLOPs Utilization (MFU) can serve as a portable, software-defined predictor of GPU power for LLMs, finding that a linear MFU-based power model fits every tested GPU as long as the workload is compute-bound, as in production LLM training.
GPU energy consumption represents a primary operational cost for Large Language Model (LLM) inference services. Despite the inherent variability of production workloads, most deployments rely on static power configurations that fail to exploit the non-linear relationship between power limits and performance. Consequent...
DEFT is presented, an energy-aware scheduling framework that jointly optimizes task-to-device assignment and per-GPU DVFS configuration for task-based multi-GPU applications and employs a cost-model–driven strategy that integrates slack awareness, throughput awareness, and explicit modeling of task execution cost, inte...
With the rapid growth of artificial intelligence (AI) and high-performance computing (HPC), GPUs and other accelerators have become a shared computing substrate for a wide range of workloads. However, many shared accelerator clusters still rely on coarse device-level allocation, which often leads to low effective utili...
A standards-based, multi-tenant cloud inference framework that integrates OpenStack orchestration with Single Root I/O Virtualization (SR-IOV)-enabled graphics processing unit (GPU) partitioning to achieve predictable and isolated real-time inference execution.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
