Sep 2026· Proceedings of the International Conference on Parallel Processing· 0 citations· 11 references
TL;DR
This work develops time and energy roofline characterizations for Orin AGX across power modes, coupled with analytical models of compute and memory for DNN inference, and extends these models to DNN training, and demonstrates power-mode tuning that achieves up to \(15\%\) lower energy with small inference-time impact for some workloads.
Abstract
Edge accelerators such as Nvidia Jetsons are an integral part of the computing continuum, and are used for DNN inference and training. Jetson edge devices have 2000+ CUDA cores within a 70W power envelope and offer 1000s of power modes to customize CPU, GPU and memory frequencies, enabling diverse power–performance trade-offs for energy-constrained deployments. While data-driven methods exist to predict power and latency, there is limited principled understanding of why different power modes perform the way they do. We develop time and energy roofline characterizations for Orin AGX across power modes, coupled with analytical models of compute and memory for DNN inference. These reveal quantitative insights, e.g., the default power mode is not the most energy-efficient and time efficiency implies energy efficiency. We extend our models to DNN training, and demonstrate power-mode tuning that achieves up to \(15\%\) lower energy with small inference-time impact for some workloads.
A controlled measurement study of self-hosted LLM inference across edge and near-edge deployment nodes: an NVIDIA Jetson AGX Orin and a near-edge server with CPU-only and GPU-enabled inference modes, highlighting that compute-side inference metrics alone can lead to suboptimal placement for latency-sensitive interactiv...
Maysam Khatib, Moysis Symeonides, Demetris Trihinas et al.· 0 citations
GreenBench, a benchmarking framework that evaluates the energy efficiency, throughput, and carbon footprint of five open-source LLMs across three NLP tasks on an Apple M4 Pro with 48 GB unified memory, is presented.
R. Kannan, Rajendra P. Firke, Shreya Bengle et al.· International Conference on...· 0 citations
This review surveys the full landscape of AI hardware accelerators for LLMs, including general-purpose GPUs, custom ASICs, reconfigurable FPGAs, processing-in-memory and near-memory architectures, and emerging neuromorphic and photonic approaches across cloud and edge deployment.
A mathematical model is proposed to predict throughput and energy consumption for concurrently executing CV workloads on edge GPU accelerators and can be integrated into functional simulation frameworks for edge–cloud deployment studies.
Abhinaba Chakraborty, D. Colle, M. Pickavet et al.· Journal of Real-Time Image P...· 0 citations
Edge platforms increasingly rely on GPUs and deep learning accelerators (DLAs) to support real-time computing of emerging workloads. However, their power, frequency, and security characteristics remain largely unexplored. This paper provides the first in-depth characterization of power and frequency behaviors of NVIDIA...
M. Rafi, Kevin Chau, Hyeran Jeon· ACM Transactions on Architec...· 0 citations
The Compatibility Ratio (CR) is introduced as a simple guideline for evaluating performance trade-offs between optimal hardware micro-architecture configurations across different workloads and shows that, for the considered accelerator, a DNN model-family optimized configuration might occupy an effective middle ground...
Lukas Groth, Andrija Nešković, Rainer Buchty et al.· ACM Transactions on Embedded...· 1 citation
Related blog posts
Microsoft Research Blog· microsoft.comOct 29, 2026
Host Sowbhagya Suresh shares some of the latest news and cutting-edge research from across Microsoft—from understanding the challenges facing coding agents to uncovering how people use AI at work and improving extended-range weather forecasting. The post Microsoft Research Focus – Edition #97 appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduOct 28, 2026
Professor Paris Smaragdis develops computers with superhuman hearing for applications like speech recognition, music understanding, and environmental monitoring.
For decades, a fundamental question about the difficulty of computing in high-dimensional spaces has resisted proof. New research resolves this question, removing a major theoretical obstacle to understanding the efficiency of randomized algorithms used across science and computing. The Kannan–Lovász–Simonovits (KLS) conjecture was proposed in 1995. It asks whether, for a broad class of high-dimensional […] The post Solving a 31-year-old grand challenge in mathematics – the KLS conjecture appear…
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.