Per-Request Paid Edge Inference for Unaffiliated Devices with Jetson Orin Nano
TL;DR
The findings show that per-request payment could be a practical way of opening idle edge accelerator computing capacity to stranger IoT devices, and to measure what that mechanism costs the edge site.
Abstract
Edge sites equipped with powerful on-device accelerators can serve low-latency inference services to IoT devices around them, and in a lot of cases, some of their capacity might be unused. In most cases, access to that unused computing power is impossible, as it would usually require a pre-created account, a dedicated API key, or a shared operator, which a lot of passing-by devices like delivery robots or inspection drones might not have. The aim of this paper is to evaluate whether an edge inference node can serve passing-by IoT devices that have no prior relationship with it, charging them for each request, without an explicit sign-up or enrollment, and to measure what that mechanism costs the edge site. For the purposes of this paper, we designed and built such a node, which can serve unknown devices, using unsettled request exposure as a resource, deployed on a three-board Jetson Orin Nano Super cluster, performing object detection on images using a YOLOv26 detector. It was tested with Raspberry Pi 5 serving as a client device, running up to 12 concurrent clients, with detailed measurements of latency and resource utilization, including an experiment with a 30-minute settlement outage. The experiments confirmed timely request processing and successful payment settlement, with reliable tracking of outstanding payments. Our findings show that per-request payment could be a practical way of opening idle edge accelerator computing capacity to stranger IoT devices.
