Compute-Aware Shared-Block Adaptive Execution for IMU-Based Human Activity Recognition

Abstract

Sensor-based human activity recognition (HAR) on edge platforms requires reducing inference-time computation while preserving recognition quality. Existing approaches, such as early exiting or explicit multibranch routing, may introduce architectural complexity or additional decision overhead. We propose a compute-aware polynomial attention drop-in replacement (PADRe) architecture that adaptively regulates per-sample computation within a shared backbone through prefix-based execution of polynomial transformations. The proposed method maintains a unified sequential path and controls the computation executed within each block according to the input, rather than changing network depth or selecting separate branches. A lightweight degree gate estimates input-dependent computation demand from a global feature descriptor. During training, the model transitions from soft routing for stable optimization to hard prefix execution aligned with inference. Each PADRe block reuses intermediate outputs to progressively construct higher order representations while avoiding unnecessary computation. Experiments on UCI-HAR, WISDM, PAMAP2, and MHEALTH show that the proposed method approaches the recognition performance of fixed maximum-degree execution while reducing average executed floating-point operations (FLOPs) and measured inference latency by approximately 19.6% and 28.5%, respectively. Matched-budget comparisons with fixed-degree variants support the effectiveness of input-adaptive execution, while controlled corruption experiments indicate relatively stable behavior under moderate signal degradation. On-device evaluation on a Raspberry Pi 4B demonstrated feasibility on a relatively high-end edge platform. Overall, the proposed framework provides a competitive recognition–computation tradeoff through input-adaptive compute control within a shared sequential backbone.