Skip to content
#edge computing Book Open access

Pagoda: Time and Energy Rooflines for DNN Workloads on Edge Accelerators Across Power Modes

Sep 2026 · Proceedings of the International Conference on Parallel Processing · 0 citations · 11 references

TL;DR

This work develops time and energy roofline characterizations for Orin AGX across power modes, coupled with analytical models of compute and memory for DNN inference, and extends these models to DNN training, and demonstrates power-mode tuning that achieves up to \(15\%\) lower energy with small inference-time impact for some workloads.

Abstract

Edge accelerators such as Nvidia Jetsons are an integral part of the computing continuum, and are used for DNN inference and training. Jetson edge devices have 2000+ CUDA cores within a 70W power envelope and offer 1000s of power modes to customize CPU, GPU and memory frequencies, enabling diverse power–performance trade-offs for energy-constrained deployments. While data-driven methods exist to predict power and latency, there is limited principled understanding of why different power modes perform the way they do. We develop time and energy roofline characterizations for Orin AGX across power modes, coupled with analytical models of compute and memory for DNN inference. These reveal quantitative insights, e.g., the default power mode is not the most energy-efficient and time efficiency implies energy efficiency. We extend our models to DNN training, and demonstrate power-mode tuning that achieves up to \(15\%\) lower energy with small inference-time impact for some workloads.

Read PDF

Similar papers

#artificial intelligence Preprint Sep 2026

A Measurement Study of LLM Inference Trade-offs Across Edge Continuum Hardware

A controlled measurement study of self-hosted LLM inference across edge and near-edge deployment nodes: an NVIDIA Jetson AGX Orin and a near-edge server with CPU-only and GPU-enabled inference modes, highlighting that compute-side inference metrics alone can lead to suboptimal placement for latency-sensitive interactiv...

Maysam Khatib, Moysis Symeonides, Demetris Trihinas et al. · 0 citations
#artificial intelligence Conference Open access Aug 2026

Greenbench: Benchmarking Energy Efficiency and Carbon Footprint of Open-Source Llm Inference on Apple Silicon

GreenBench, a benchmarking framework that evaluates the energy efficiency, throughput, and carbon footprint of five open-source LLMs across three NLP tasks on an Apple M4 Pro with 48 GB unified memory, is presented.

R. Kannan, Rajendra P. Firke, Shreya Bengle et al. · 0 citations
Review Aug 2026

AI Hardware Accelerators for Large Language Models: Architectures and the Memory Wall

This review surveys the full landscape of AI hardware accelerators for LLMs, including general-purpose GPUs, custom ASICs, reconfigurable FPGAs, processing-in-memory and near-memory architectures, and emerging neuromorphic and photonic approaches across cloud and edge deployment.

Siddharth Patel, Rohit Singh · 0 citations
Aug 2026

Modeling throughput and power consumption for real-time concurrent vision application deployments on edge

A mathematical model is proposed to predict throughput and energy consumption for concurrently executing CV workloads on edge GPU accelerators and can be integrated into functional simulation frameworks for edge–cloud deployment studies.

Abhinaba Chakraborty, D. Colle, M. Pickavet et al. · 0 citations
Open access Aug 2026

Uncovering Power and Frequency Behaviors and Their Security Implications in Edge GPU Platforms

Edge platforms increasingly rely on GPUs and deep learning accelerators (DLAs) to support real-time computing of emerging workloads. However, their power, frequency, and security characteristics remain largely unexplored. This paper provides the first in-depth characterization of power and frequency behaviors of NVIDIA...

M. Rafi, Kevin Chau, Hyeran Jeon · 0 citations
#edge computing Sep 2026

Compatibility Ratio as a Guideline for Hardware/DNN Co-Design of Embedded Accelerators

The Compatibility Ratio (CR) is introduced as a simple guideline for evaluating performance trade-offs between optimal hardware micro-architecture configurations across different workloads and shows that, for the considered accelerator, a DNN model-family optimized configuration might occupy an effective middle ground...

Lukas Groth, Andrija Nešković, Rainer Buchty et al. · 1 citation

Related blog posts

Microsoft Research Blog Oct 29, 2026

Microsoft Research Focus – Edition #97

Host Sowbhagya Suresh shares some of the latest news and cutting-edge research from across Microsoft—from understanding the challenges facing coding agents to uncovering how people use AI at work and improving extended-range weather forecasting. The post Microsoft Research Focus – Edition #97 appeared first on Microsoft Research.

MIT News · Artificial Intelligence Oct 28, 2026

Making machines that can listen

Professor Paris Smaragdis develops computers with superhuman hearing for applications like speech recognition, music understanding, and environmental monitoring.

Microsoft Research Blog Oct 21, 2026

Solving a 31-year-old grand challenge in mathematics – the KLS conjecture

For decades, a fundamental question about the difficulty of computing in high-dimensional spaces has resisted proof. New research resolves this question, removing a major theoretical obstacle to understanding the efficiency of randomized algorithms used across science and computing. The Kannan–Lovász–Simonovits (KLS) conjecture was proposed in 1995. It asks whether, for a broad class of high-dimensional […] The post Solving a 31-year-old grand challenge in mathematics – the KLS conjecture appear…

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.