Knowledge Distillation and Quantization-Aware Compression for Remaining Useful Life Prediction on Resource-Constrained IoT Devices

Abstract

Models that achieve the best desktop accuracy in Remaining Useful Life (RUL) prediction do not necessarily meet the deployment constraints on IoT microcontrollers. The dominant evaluation paradigm rewards offline accuracy, yet condition-based maintenance ultimately requires inference within strict memory and latency budgets. This paper addresses this gap by evaluating three composable compression mechanisms, namely matched-capacity architecture scaling, Knowledge Distillation (KD), and INT8 Quantization-Aware Training (QAT), on the C-MAPSS FD001 benchmark with ESP32-S3 hardware validation. The analysis shows that the temperature-scaled KD is mathematically inert for scalar regression: KD improves cross-seed consistency (3.5× variance reduction) more than mean accuracy. Matched-capacity scaling compresses a 147K-parameter TTSNet teacher into a 33K StudentTTSNet (77.7% reduction), and INT8 QAT adds a 3.98× weight-level reduction, yielding a 70.9 KB deployed model at only 1.6% RMSE cost. On the ESP32-S3 microcontroller, the best offline GRU baseline fails to produce correct predictions in the tested TFLite Micro configuration due to the 1024-tensor allocation limit, whereas the compressed student model runs in real time at 56.8 ms (FP32) and 83.9 ms (INT8). Preliminary cross-domain experiments on a small plant-extraction dataset suggest that the same pipeline can regularize smaller data settings, indicating that runtime compatibility deserves validation as early as offline accuracy in IoT-bound RUL research.