Fast and Accurate Classification with Parallel IDK Classifier Cascades

Deployment method of deep object detection models integrating model pruning and data reuse for reconfigurable architectures

Achieving Near-Zero-Overhead Multi-Model Hierarchical Classification in Real-Time Detection Pipelines

Edge-deployed vision systems in target recognition, surveillance, autonomous vehicles, and drone domains require hierarchical inference pipelines where a detection model identifies objects of interest and downstream classifiers provide fine-grained attribute analysis. Running all models on the GPU creates a serial bottleneck that limits real-time throughput as pipeline stages grow. Modern edge SoCs pair GPUs with dedicated neural accelerators (NPUs, DLAs) capable of concurrent execution, yet deploying custom models on these accelerators remains impractical due to strict operator constraints, quantization incompatibilities, and an undocumented end-to-end pipeline. We target NVIDIA Jetson DLA cores as the representative platform. We present a five-step methodology for zero GPU fallback DLA INT8 deployment of classification backbones, comprising architecture adaptation, manual dynamic range workaround to rescue TensorRT's implicit quantization (recovering 94.0% accuracy from implicit quantization's 75%) for rapid pipeline validation before explicit quantization, quantization-aware training, ONNX graph surgery for DLA compilation, and a concurrent GPU-detection/DLA-classification inference pipeline. We document nine engineering constraints with root-cause analysis and generalizable solutions. Validation on a dual-head person attribute classifier running on DLA alongside a GPU object detector on a Jetson Orin NX demonstrates near-zero pipeline overhead (12.5 vs. 13.3~FPS detector-only at 1080p), with dual-DLA scaling at no additional cost. The methodology is backbone-agnostic and generalizes to any detection-classification edge pipeline.

Optimizing MNIST Digit Recognition for Edge Devices: A Multi-dimensional Evaluation Approach

The authors introduce the Edge Suitability Score (ESS), a composite metric that combines normalized accuracy, model size, and inference speed into a single value, weighted at 0.40, 0.35, and 0.25 to reflect their relative importance for microcontroller deployment.

Predicting CNN training latency on cloud GPUs without disclosing model architecture

ERASE: EaRly bAckpropagation SchEdule for Faster Training of Modern Recommendation Systems

The detachment mechanism of Forward-Forward (FF) is reinterpreted as a scheduling primitive: given a local objective, detaching a block's output removes downstream gradient dependencies, making its backward pass ready when its forward pass finishes.

Automatic Model Compression and Quantized Deployment of Convolutional Neural Networks on Programmable Data Planes

The rapid development of programmable network devices and the widespread adoption of machine learning (ML) in networking have facilitated efficient research into intelligent data planes (IDPs). Offloading ML to programmable data planes (PDPs) enables quick analysis and responses to network traffic dynamics, and efficient management of network links. Compared to using an external low-cost board with sufficient memory and a general-purpose CPU, IDP deployment keeps inference inside the switch forwarding pipeline, avoiding inter-device transfer and coordination overhead. This enables line-rate processing and faster response for real-time network control. However, the hardware pipeline presents significant resource limitations. For instance, Intel Tofino ASIC has only 10Mb SRAM in each stage, and lacks support for multiplication, division, and floating-point operations. These constraints significantly hinder the development of IDP. This paper presents Quark, a framework that automatically compresses the convolutional neural network (CNN) and fully offloads quantized inference onto PDP. Quark employs model pruning to simplify the CNN model, uses quantization to support floating-point operations, and utilizes neural architecture search to balance accuracy and PDP resource constraints. Additionally, Quark divides the CNN into smaller units to improve resource utilization on the PDP. We have implemented a testbed prototype of Quark on both P4 hardware switch (Intel Tofino ASIC) and software switch (i.e., BMv2). Extensive evaluation results on the ISCX Botnet dataset demonstrate that Quark achieves 97.3% accuracy while using only 24.27% of the SRAM resources on the Intel Tofino ASIC switch, completing inference tasks at line rate with an average latency of $42.66\mu s$ .