Jul 2026· Fall Joint Computer Conference· pp. 339-344· 0 citations· 11 references
Abstract
With the explosive development of LLM-empowered agent technology, LLM inference performance has become more important than training. Cloud computing is a popular deployment approach, where performance prediction is vital for instance selection and QoS assurance. However, prediction is challenging due to GPU hardware heterogeneity, Transformer operator variations, and dynamic inference configurations. Virtualization and other features vary across clouds, further increasing prediction difficulty. Existing methods suffer from low accuracy and poor generalization. To tackle these issues, we propose Dispeller, a prediction model for GPU-accelerated cloud environments with three feature sets: 1) basic GPU hardware feature with 7 dimensions; 2) operator-level GPU performance feature with 4 dimensions; 3) inference configuration feature with 4 dimensions. We conduct experiments on public cloud GPUs and collect a real-world dataset of 10,112 samples. Random Forest is adopted to learn the nonlinear mapping between features and performance. Experimental results show that Dispeller achieves high prediction accuracy with TPS $\mathrm{R}^{{2}} = 0.951$ on seen GPUs and 0.989 on unseen GPUs, demonstrating strong cross-GPU generalization. An ablation study confirms that inference configuration features contribute 87.3% of the predictive power. Dispeller is therefore able to recommend cloud resources and optimize LLM deployment costs.
A dynamic recurrent neural network is proposed to accurately predict workloads and integrates an auto-encoder to effectively extract representations from the original workload data with high dimensionality to enable adaptive and accurate predictions for highly variable workloads.
Okore Kalu, C. Okafor, P. Asuquo et al.· E3S Web of Conferences· 0 citations
A deep learning-based failure prediction model that integrates Convolutional Neural Networks and Bidirectional Long Short-Term Memory networks to identify job failures before they occur is presented, improving the performance of cloud computing applications by reducing job failures and optimising resource utilisation.
Wunukhen Shehu Awudu, P. Asuquo, B. Agbor et al.· E3S Web of Conferences· 0 citations
Computation methods of artificial intelligence are gradually shifting from cloud computing to edge computing and on-device machine learning (ODML). How to contribute an effective machine learning model in the resource-limited environment, has become a significant and rapidly evolving research field. The training and inference of deep learning model used to be performed on the cloud high-performance computing clusters. There are many problems with uploading data to cloud, for example, high latency, round-trip latency, security issues, and a lack of privacy guarantees, and in this case, people cannot make real-time decisions. So, using edge devices to process tasks can significantly decrease the cost of transmission. The need of low latency, quick response, privacy protection and high adaptability has become the drive force of this change. This report aims to provide a comprehensive overview of lightweight model inference technologies in edge computing environments, mainly targeting low performance devices, such as mobile phones, intelligent equipment in vehicle, VR/AR headsets and Internet of Things (IoT). This paper introduces efficient learning and inference on edge devices from four aspects: 1) the definition of core terminology and concrete application environment; 2) the core technology of model compression, neural networks and knowledge distillation, which is used to deal with the tasks in the resource-limited environment; 3) the standards for evaluation of time/space complexity; 4) the challenge and opportunity which people face currently and in future.
Hengrui Bi· Applied and Computational En...· 0 citations
The growing demand for ultra-low-latency, high-throughput services (with 5G networks) will succeed the (proliferation of 5G networks) multi-access edge computing (MEC). Thus enabling the next phase of communication systems. In a 5G MEC architecture, the choice of when to offload a computation in an Edge Cloud server or Cloud server has serious implications for latency, resource use and quality of service. In this paper, we compare six approaches to this server-selection decision: a Random Forest, XGBoost, Support Vector Machine, Gradient Boosting, an Artificial Neural Network, and a decision engine based on a Large Language Model (Claude, Anthropic). All six methodologies were put to assess on a common, reproducible pipeline from exploratory data analysis to feature engineering to stratified data partitioning and feature standardisation, utilising the same dataset of 35,000 task instances of network and server telemetry. All of the five trained machine learning and deep learning models achieved near-optimal classification performance. The use of a three-pronged leakage diagnostic called feature-importance analysis, cross-tabulation, and ablation experiments revealed that the models relied on informative distance-linked features as opposed to inherent model capability. In terms of a no-leakage feature subset, the XGBoost model, which is the strongest classical model, achieved an accuracy of 99.70% while the Claude-based LLM engine achieved 96.0% accuracy on a noleakage comparable subset. In line with the requirement that using an existing LLM does not, by itself, demonstrate original contribution, we further propose and evaluate a Hybrid Confidence-Gated Decision Engine based on XGBoost and Claude, which escalates only cases deemed genuinely uncertain to the LLM. This hybrid engine obtained an overall accuracy of 99.90% but failed to beat the XGBoost rejected predictions on the very few escalated cases a negative result reported honestly. Meanwhile, a deeper diagnostic demonstrated that every field in our dataset, beyond the original four that we flagged, was strongly correlated with the target label. The data leakage and hybrid decision systems are examined in this multi-access edge computing assisted server selection using machine learning in 5G networks research article by Claude and more.
Amandeep, Ankit, Dharmender Kumar et al.· International Journal of Sci...· 0 citations
Predictive maintenance in smart factories requires not only high prediction accuracy but also low-latency processing, adaptability to data drift, and understandable explanations for field operators. However, many existing approaches remain cloud-centered, label-dependent, and weak in practical explainability. This study proposes an explainable predictive maintenance framework based on FPGA-GPU Edge–Cloud hybrid computing for large-scale multivariate time-series environments. In the proposed system, FPGA modules perform streaming-oriented signal preprocessing and low-latency feature extraction, while GPU modules execute deep learning-based anomaly detection and fault prediction. To reduce dependence on labeled fault data, the framework incorporates masked autoencoder-based self-supervised representation learning. To improve long-term robustness in changing manufacturing environments, the framework also considers continual learning based on Elastic Weight Consolidation. In addition, a lightweight large language model with parameter-efficient fine-tuning and retrieval-augmented generation generates root-cause-oriented explanations and maintenance guidance. The method is organized as an integrated pipeline that combines data acquisition, edge preprocessing, temporal inference, explanation generation, and cloud-assisted model adaptation. The evaluation framework includes certification-oriented testing, comparative analysis, latency and throughput analysis, drift response analysis, and explainability assessment. According to a third-party test report, the proposed system achieved an event recall of 0.9822, event precision of 0.9529, event F1-score of 0.9674, and a false alarm rate of 0.000205. These results indicate that the proposed framework is practically feasible for real-time, explainable, and deployable predictive maintenance in smart factory environments.
Hyunwook Kim, Ki-Tae Kim, Se-Jin Lee et al.· International Journal on Adv...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.