Jul 2026· IEEE International Conference on Cloud Computing· pp. 68-77· 0 citations· 22 references
Abstract
Modern web services are expected to meet key non-functional requirements—such as availability, responsiveness, scalability, and reliability—typically formalized through Service Level Agreements (SLAs). These agreements specify Service Level Objectives (SLOs), including metrics such as latency, throughput, and uptime, to guarantee consistent service quality. Failure to meet these targets can result in financial penalties and reputational damage, while excessive resource provisioning leads to wasted costs and inefficiencies.Autoscaling techniques help address this challenge by dynamically adjusting the number of service replicas based on demand. However, traditional autoscaling methods mainly depend on low-level indicators such as CPU and memory usage, limiting their effectiveness in balancing SLO compliance with cost efficiency. This paper introduces an enhanced feedback-control-based autoscaling approach for containerized applications, specifically tailored for Kubernetes environments, which directly incorporates response time SLOs into scaling decisions. By doing so, the proposed method enhances scaling accuracy and achieves a better trade-off between performance and resource utilization. Experimental results from a prototype implementation show significant improvements over the standard Kubernetes Horizontal Pod Autoscaler and another autoscaling methodology proposed in literature.
Modern web services are required to meet critical non-functional requirements, including availability, responsiveness, scalability, and reliability, which are formalized through Service Level Agreements (SLAs). SLAs define Service Level Objectives (SLOs), such as latency, throughput, and uptime, that ensure consistent service quality. Failing to meet these objectives can incur penalties and harm a provider's reputation. At the same time, over-provisioning resources leads to unnecessary costs and inefficient utilization. Autoscaling mechanisms address this by dynamically adjusting the number of service replicas according to demand. However, conventional approaches typically rely on low-level metrics, such as CPU or memory usage, which limit the ability to optimize both SLO compliance and infrastructure costs. This paper presents an enhanced SLO-driven autoscaling methodology for containerized workloads in Kubernetes clusters, integrating response time SLO targets into the autoscaling process. The proposed approach improves decision-making over traditional autoscaling by balancing service-level performance with operational efficiency. Experimental evaluation of a prototype demonstrates clear benefits compared to the default Kubernetes Horizontal Pod Autoscaler.
A. Marchese, O. Tomarchio· International Conference on...· 0 citations
Autoscaling microservice-based applications to satisfy Service Level Objectives (SLOs) remains challenging due to bursty workloads, cascading latency across service dependencies, and cold-start overhead. Existing approaches such as the Kubernetes Horizontal Pod Autoscaler (HPA) rely on threshold-based CPU or memory metrics, which react too slowly to traffic spikes. Recent predictive methods improve responsiveness but generate point forecasts that ignore prediction uncertainty, leading to over-provisioning or oscillatory scaling. We propose SLO-Scaler, an uncertainty-aware autoscaling framework that predicts short-horizon request rates, tail latency, and SLO violation probability using a Bayesian LSTM model. SLO-Scaler integrates confidence-interval-based scaling decisions with a dependency graph analysis module that localizes bottleneck services, avoiding unnecessary whole-chain scaling. We evaluate SLO-Scaler on the DeathStarBench Social Network benchmark deployed on Kubernetes under periodic, bursty, and long-tail traffic patterns. Under bursty traffic, SLO-Scaler reduces the SLO violation rate by 29-56%, lowers the average replica count by 18-33%, and decreases scaling event frequency by 38-59% compared with the baselines, while achieving lower tail latency.
Shuo Wang, Xiaoxuan Sun, Shao-yu Huang et al.· 0 citations
Static threshold-based autoscaling with Kubernetes Horizontal Pod Autoscaler (HPA) does not cover asynchronous event-driven workloads that only target CPU and memory usage average metrics, such as the default HPA does. The HPA targets metrics that are lagging indicators‚ unlike the queue depth and stream backlog metrics‚ and thus introduces SLO violations in document processing and data integration services․ This paper describes a production-qualified implementation for optimizing KEDA (Kubernetes Event-Driven Autoscaling) and Karpenter to manage and optimize AWS EKS pods and nodes․ KEDA extends the Kubernetes Horizontal Pod Autoscaler (HPA) for external scalers, allowing workloads' Kubernetes pods to scale based on SQS queue depth, DynamoDB Streams shard lag, CloudWatch metrics, and Prometheus time series. When combined with KEDA, Karpenter can also provision nodes from heterogeneous EC2 Spot Instance pools and consolidate nodes when underutilized. Overall, customers who have deployed Karpenter on production EKS clusters report a 40-60% reduction in compute costs and SLO adherence on variable workloads over time, as compared to EC2 On-Demand Node Groups. SLO-aware scheduling results in a 31% reduction in SLO violations and an 18% reduction in infrastructure cost. Scaling to zero (ignoring the cost of idle compute resources when there is no demand for workloads) and Spot diversification (up to 44%) optimize compute cost for off-peak workloads. Deployment patterns‚ configuration‚ graceful termination‚ and operational observability requirements are described with a focus on production deployments in enterprise environments․
Avneet Bansal· International journal of com...· 0 citations
OpScale is presented, a practical operator-level orchestration framework of profiling, provisioning, placement, and runtime serving that attains SLOs with up to 36.3% fewer GPUs and 28% less power, or achieves 44% higher throughput under fixed cost budgets.
Xingqi Cui, Chieh-Jan Mike Liang, Ziang T. Tang et al.· 0 citations
Kubernetes is the standard platform for orchestrating containerized applications, yet resource management remains difficult. To stay safe, engineers over-provision CPU and memory, leaving reserved but unused capacity that is the main source of wasted cost. The built-in Horizontal and Vertical Pod Autoscalers are reactive: they act only after a threshold is crossed, which causes lag, over-provisioning, and can mask software defects by granting a leaking workload more memory. Predictive autoscalers focus on improving forecasting accuracy or run inside proprietary infrastructure, and anomaly detection is used only to alert, never to block a harmful action. The Intelligent Cluster Optimizer is an open-source Kubernetes operator that right-sizes container workloads with safety as a first-class concern. Its central contribution is a five-layer safety pipeline where a memory-leak detector, based on linear regression with R^2 scoring, acts as a blocking gate: if a leak is detected the recommendation is rejected, so the optimizer never hides a bug by enlarging a broken container. The pipeline combines SLA monitoring, a circuit breaker, HPA/PDB conflict detection, and a policy engine, with rollback and dry-run mode for human approval. Recommendations are produced by percentile analysis and Holt-Winters forecasting, balanced through multi-objective Pareto optimization at the per-container level. We validated the system with 1118 automated tests at 80.3% coverage and a live deployment on Google Kubernetes Engine, where right-sizing produced estimated cost savings of 20--40% in what-if projections and the leak gate reached 83% detection accuracy.
Azra Karakaya, Erva Şengül, Ahmet Kaplan· arXiv.org· 0 citations
Cloud-edge controllers coordinate service placement, replica scaling, and resource pre-warming to keep end-to-end latency within application deadlines. But evaluations often obscure the source of a reported gain: placement and scaling are studied separately; workload, connectivity, and calibration assumptions remain implicit; and metrics over completed tasks hide unfinished work. We present ContinuumBench, a benchmark that controls these factors. Its completion-aware accounting treats late, unfinished, and discarded tasks as deadline misses. A common protocol compares placement-only and scale-capable controllers under declared regimes and stressors. Built on the ECLYPSE simulator, ContinuumBench adds arrivals, worker elasticity, intermittent transport, buffering, and failures to close the control loop. We evaluate nine controllers across four scenarios and two regimes. The studied regimes are capacity-bound: elastic capacity, not placement sophistication, drives completion, and once capacity suffices, the choice of autoscaling policy decides how much of that work arrives on time. Placement re-planning has no measurable effect without relocation, while cost-free migration defines the observed exception. Consequently, scale-capable controllers approach an over-provisioned reference while placement-only controllers degrade with load; and placement quality separates controllers only once capacity is exhausted. Finally, the accounting choice itself changes the reported result: completion-only and completion-aware scoring can rank controllers differently.
Unknown authors· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
