Experiments show that, after applying the proposed knowledge distillation method, the inference latency is reduced from 235 ms to a minimum of 26 ms, which is better than DistilBERT’s 35 ms, verifying the efficiency and practicality of the lightweight model in resource-constrained scenarios involving power-risk identification, electromagnetic sensing, and edge-based intelligent monitoring.
Abstract
Large language models demonstrate high precision in power-system risk identification; however, their massive parameter counts and high resource consumption hinder real-time deployment on resource-constrained edge devices used in electromagnetic sensing systems, wearable monitoring terminals, and compact power-monitoring devices. Achieving low-latency processing in distributed sensing structures and edge-based electromagnetic monitoring devices requires a significant reduction in computational overhead to ensure immediate detection of electrical hazards, abnormal equipment states, and potential power-system risks. This paper proposes a lightweight compression method based on hierarchical supervised knowledge distillation. Experiments show that, after applying the proposed knowledge distillation method, the inference latency is reduced from 235 ms to a minimum of 26 ms, which is better than DistilBERT’s 35 ms. The number of student model parameters is reduced to 4.3% of the teacher model, namely 14.5M versus 340M, while the classification accuracy reaches 89.4%, close to the teacher model’s 92.7%. The F1 score for the equipment failure category reaches 90.3%, verifying the efficiency and practicality of the lightweight model in resource-constrained scenarios involving power-risk identification, electromagnetic sensing, and edge-based intelligent monitoring.
The rapid expansion of Large Language Model (LLM) serving in cloud data centers has created a critical need for energy-aware scheduling. However, estimating inference energy typically requires hardware-level power telemetry, which is rarely accessible to cloud tenants. This paper proposes a lightweight, machine-learning-based framework for estimating the energy consumption and CO2 emissions of LLM inference using only software-observable runtime features. We collected 4,950 inference records across five model architectures in a Vertex AI environment, evaluating varying batch sizes and simulated demand levels. We demonstrate that a Random Forest regressor can accurately predict inference energy (R2 = 0.994) without requiring privileged hardware instrumentation. Feature-importance analysis reveals that inference latency accounts for 76.7% of the model’s predictive variance, far outweighing token count as the primary driver of energy cost. To demonstrate practical utility, we implement an interactive prototype for real-time energy estimation. This work establishes latency as a highly reliable proxy for energy, providing system designers with an actionable metric for sustainable LLM deployment.
Bediga Sharan, Swarup Ghosh· 2026 4th International Confe...· 0 citations
Models that achieve the best desktop accuracy in Remaining Useful Life (RUL) prediction do not necessarily meet the deployment constraints on IoT microcontrollers. The dominant evaluation paradigm rewards offline accuracy, yet condition-based maintenance ultimately requires inference within strict memory and latency budgets. This paper addresses this gap by evaluating three composable compression mechanisms, namely matched-capacity architecture scaling, Knowledge Distillation (KD), and INT8 Quantization-Aware Training (QAT), on the C-MAPSS FD001 benchmark with ESP32-S3 hardware validation. The analysis shows that the temperature-scaled KD is mathematically inert for scalar regression: KD improves cross-seed consistency (3.5× variance reduction) more than mean accuracy. Matched-capacity scaling compresses a 147K-parameter TTSNet teacher into a 33K StudentTTSNet (77.7% reduction), and INT8 QAT adds a 3.98× weight-level reduction, yielding a 70.9 KB deployed model at only 1.6% RMSE cost. On the ESP32-S3 microcontroller, the best offline GRU baseline fails to produce correct predictions in the tested TFLite Micro configuration due to the 1024-tensor allocation limit, whereas the compressed student model runs in real time at 56.8 ms (FP32) and 83.9 ms (INT8). Preliminary cross-domain experiments on a small plant-extraction dataset suggest that the same pipeline can regularize smaller data settings, indicating that runtime compatibility deserves validation as early as offline accuracy in IoT-bound RUL research.
In intelligent power systems operating under increasingly complex electromagnetic environments, accurate identification of safety risk points is essential for ensuring reliable equipment operation and supporting electromagnetic compatibility assessment. Traditional rule-based methods suffer from limited semantic understanding and poor generalization, making them insufficient for processing complex operation and maintenance texts. To address this issue, this paper proposes a lightweight CNN-based pre-training model built on a BERT-Free architecture for efficient and accurate power safety risk identification. A professional dataset containing 58,000 power operation and maintenance texts is constructed, and a high-quality multi-label corpus covering four categories and 17 subcategories is established through dictionary-enhanced word segmentation and expert cross-annotation. The model is exported in ONNX format to facilitate flexible deployment in engineering applications. Experimental results demonstrate that the proposed model achieves an overall accuracy of 89.1% and an F1-score of 87.3%. Under class imbalance, the F1-score reaches 0.901 for majority classes and 0.784 for minority classes, exhibiting strong robustness. The average inference time per sample is only 8.3 ms, indicating high computational efficiency. These results demonstrate that the proposed model provides an effective and practical solution for intelligent power safety risk identification while offering valuable support for electromagnetic infrastructure monitoring and reliable operation in modern power systems.
S. W. Yu, Y. He, G. Ban et al.· Advanced Electromagnetics· 0 citations
Experimental results demonstrate that FMDF significantly reduces model complexity while maintaining high predictive accuracy, scalability, robustness, and energy efficiency, making it a promising solution for resource-efficient predictive AI, edge intelligence, federated learning, digital twins, and next-generation intelligent decision support systems.
Ken Iverson· International Journal of Mac...· 0 citations
Long short-term memory (LSTM) networks have been widely applied to battery state-of-charge (SOC) estimation because of their capability to capture nonlinear battery dynamics and long-term temporal dependencies. However, deploying high-accuracy LSTM-based SOC estimation models on resource-constrained edge devices remains challenging, as conventional model compression methods often disrupt the gated structures responsible for temporal dependency modeling, resulting in degraded estimation accuracy. To address this issue, a lightweight SOC estimation method, termed Gate-Aware Pruned LSTM Enhanced by Knowledge Distillation (GAP-LSTM-KD), is proposed. A first-order Butterworth filter is employed to suppress measurement noise. A gate-aware pruning strategy is developed to evaluate hidden-unit importance by jointly considering gate weights and gradient sensitivity, enabling structural compression while preserving critical temporal information. Knowledge distillation is further introduced to compensate for the representation loss caused by pruning. Experimental results show that, at a 40% pruning rate, the proposed model reduces edge inference latency by more than 50%, while parameter count and computational cost are reduced by over 60%. The Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) remain no higher than 0.76% and 0.60%, respectively. The proposed method maintains competitive SOC estimation accuracy across different temperatures and operating conditions while substantially improving computational efficiency.
Wen-Qiang Huang, Ting He, Wen-Long Zhu· Journal of the Electrochemic...· 0 citations
Experimental results validate the feasibility of transferring knowledge from large models under low-computational-power constraints and provide a new technical pathway and engineering reference for recognising anomalies at the edge of intelligent connected vehicles.
Jian-Jun Zeng, Jianguo Wei, Ge Song· 電腦學刊· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
