Back to feed
Open access

EMC+: An Opportunistic Elasticity Method for Improving System Throughput and CPU Utilization in Cloud Data Centers

Aug 2026 · Concurrency and Computation · Vol 38 · 0 citations · 51 references

TL;DR

The new EMC+ proposal is an OS‐driven elasticity manager for container‐based environments that continuously estimates idle core cycles left by regular (inelastic) applications, and reallocates idle cores to elastic ones, even during short time intervals, and has minimal impact on the performance and QoS of colocated inelastic applications.

Abstract

Multicore architectures have become the prevailing processor design for general‐purpose computing systems, enabling high performance across a wide range of commercial platforms. Technological advances have made it possible to integrate hundreds of cores on a single package, providing unprecedented computational capacity. In cloud data centers, efficiently exploiting this growing core count becomes crucial for improving resource efficiency, lowering operational costs, and boosting revenue. To achieve this, servers usually run multiple colocated cloud services alongside diverse workloads. However, fully utilizing all available cores while enforcing quality‐of‐service (QoS) constraints for critical services remains a major challenge. This paper introduces a novel OS‐level approach designed to maximize CPU usage in multicore servers through the opportunistic acceleration of elastic HPC workloads. These workloads are capable of dynamically adjusting the number of active worker threads/processes at runtime. Our proposal, referred to as EMC+, is an OS‐driven elasticity manager for container‐based environments that continuously estimates idle core cycles left by regular (inelastic) applications, and reallocates idle cores to elastic ones, even during short time intervals. The new EMC+ proposal builds on our previous work and broadens the scope of OS‐driven elasticity through several key contributions: the design of a new kernel‐level container‐management framework, efficient operation on large‐scale multicore platforms, the integration of QoS concerns into elasticity exploitation, and a comprehensive experimental analysis using diverse workloads that combine cloud services and HPC applications based on different parallel programming models. Experimental results show that EMC+ speeds up elastic workloads by up to 2.3 ×$$ \times $$ (1.5 ×$$ \times $$ on average), while having minimal impact on the performance and QoS of colocated inelastic applications. Compared to the unmodified Linux kernel, which does not exploit opportunistic elasticity, EMC+ boosts average system throughput by 19%, while degrading inelastic application performance by only 2.2% on average, and increasing the occurrence of observation intervals with QoS violations by 3.1% on average.

Read PDF

Similar papers

Review Open access Jul 2026

Enhancing the Kubernetes Scheduler: A State-of-the-Art Review from Cloud to Edge

A comprehensive review of Kubernetes scheduling strategies published between January 2023 and January 2026 is presented and a multi-dimensional taxonomy is established that categorizes scheduling approaches based on common objectives, modification methods, optimization methodologies, targeted workloads, evaluation methods, scheduling scopes, and performance metrics.

Mohammed Alhakimi, R. Latip · 0 citations
Jul 2026

Evaluating Paramount Iterations as a Performance Proxy for High Performance Computing Applications in the Cloud

Cloud computing has transformed data storage, access, and processing, offering significant opportunities and advantages. However, selecting the most cost‐effective and efficient cloud resources for high‐performance computing (HPC) workloads remains a challenge, due to their long‐running and resource‐intensive characteristics that require tailored allocation strategies. In this work, a methodology is presented to optimize cloud resource selection for HPC applications using a performance proxy based on a few Paramount Iterations (PIs). The proxy enables the estimation of Relative Performance and Relative Cost between two cloud configurations with minimal overhead, allowing different optimization strategies to identify configurations that minimize execution time or cost. This approach extends previous work by evaluating a broader range of applications and cloud configurations, systematically identifying limitations and highlighting scenarios where the technique is most effective. The methodology was evaluated across three cloud providers, and results show that selecting instances based on the second PI achieves at least 97% of the best achievable execution time in most cases, while highlighting cases where additional PIs improve selection accuracy. These findings confirm that the proposed methodology effectively predicts optimal cloud resources for HPC workloads, reducing verification costs and providing a practical framework for informed resource selection.

J. R. Brunetta, J. Borin, E. Borin · 0 citations
Conference Jun 2026

Release of a Comprehensive Dataset for Fostering Dynamic Power Management Mechanisms in the Edge-Cloud Continuum

The rapid development and scaling of mobile telecommunications networks, together with related domains such as the edge-cloud continuum have raised significant concerns regarding energy consumption and environmental sustainability. Addressing these concerns requires a focus on CPU energy consumption, as CPUs are among the largest energy consumers in these systems. This paper investigates existing techniques, with a focus on CPU idle states (C-states), performance states (P-states), and frequency scaling governors implemented at both hardware and software levels. These mechanisms enable the dynamic adjustment of CPU parameters, providing opportunities to optimize power consumption, frequency, voltage, and overall system performance. In this regard, three CPUs with different architectures from well-known manufacturers, Intel® and AMD®, are thoroughly examined. A comprehensive dataset, collected under three load scenarios (idle, medium, and high), is used to support the analysis, reflect realistic runtime conditions, and enable a comparison of the technological differences in how these parameters are exposed and utilized.

M. Akbari, R. Bolla, R. Bruschi et al. · 0 citations
Open access Aug 2026

CELLServe: An SLO-Aware and Cost Efficient LLMs Serving System for Serverless Computing Environments

CELLServe formalizes SLO-constrained joint resource provisioning as an optimization problem with a dedicated algorithm, and introduces an opportunistic instance merging strategy for decode phase functions to reclaim fragmented resources.

Zejian Wang, Nan Lin, Zinuo Cai et al. · 0 citations
Open access Jun 2026

Performance modeling in FaaS workloads at the edge using machine learning

The Function as a Service (FaaS) paradigm has emerged as a compelling architectural model for both cloud and edge computing environments, enabling the execution of self-contained functions triggered by specific events while abstracting from developers infrastructure management complexities such as load balancing and auto-scaling. In FaaS-enabled clusters, particularly within resource-constrained edge environments, precise resource consumption estimation becomes critical to optimize resource utilization, minimize latency, prevent system overloads, and ensure scalability. This paper addresses performance modeling challenges in FaaS-enabled distributed and decentralized edge computing systems, operating at the granularity level of both nodes and individual functions. We propose a Machine Learning-based framework designed to predict key performance indicators, including CPU utilization, memory, and energy consumption, based on incoming workload patterns, while simultaneously forecasting potential system overload conditions. Moreover, our approach introduces a profiling methodology that characterizes serverless functions according to their resource consumption profiles, thereby enabling accurate prediction of node-level resource demands without requiring detailed knowledge of individual deployed functions. Experimental validation demonstrates that our predictive models achieve 97% accuracy in anticipating node overload scenarios, providing a robust foundation for proactive resource management in edge-based FaaS deployments. Moreover, while our best individual, function-based regression models predict node-level CPU, RAM and power consumption with a Mean Absolute Percentage Error below 9% on average, our experiments highlight the effectiveness of function profiling and cluster-based modeling. On one hand a novel multi-target regressor based on a permutation-invariant neural architecture is proved to generalize effectively across previously unseen workload compositions in the tested scenarios, with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$R^2$$\end{document} scores ranging from 0.94 to 0.98, i.e., aligned with the performance of specialized single-target regression models. On the other hand, cluster-level models generalize effectively to previously unseen functions within the same usage class, maintaining prediction errors within practically acceptable ranges and often in the single-digit percentage range.

Federica Filippini, Marco Savi, Michele Ciavotta · 0 citations