AutoSLO is presented, a latency-SLO-aware workload management framework for multi-cluster cloud data warehouses that successfully meets latency SLOs of varying strictness, reducing cost by a mean of 26.4% compared to the per-scenario next-best baseline.
Abstract
Modern cloud data warehouses decouple compute from storage, making it easy for organizations to access the same underlying data with multiple compute clusters. This flexibility is often used for performance isolation among diverse workloads, so that each workload meets its latency service-level objective (SLO) more reliably. For example, interactive dashboards, ad hoc analysis, and batch jobs can each run on separate clusters. However, this dedicated-cluster approach requires each compute cluster to be continuously scaled to adapt to workload evolution, with over-provisioning wasting resources and under-provisioning risking SLO violations. We present AutoSLO, a latency-SLO-aware workload management framework for multi-cluster cloud data warehouses. AutoSLO operates across three timescales through three key components. First, a periodic Policy Tuner plans proactive cluster scaling actions and tunes configuration parameters, using simulations of history-derived workload forecasts. Second, an SLO-aware reactive Autoscaler adjusts the active cluster set when recent workload behavior deviates from the forecast. Third, an online Query Router reacts to live load when placing each query, using a concurrency-aware latency predictor to avoid SLO violations. On realistic Redbench workloads, AutoSLO successfully meets latency SLOs of varying strictness, reducing cost by a mean of 26.4% compared to the per-scenario next-best baseline. Component-level evaluations show that the Query Router and Autoscaler respectively reduce SLO violation rates by a mean of 47.8% and 93.7%, relative to their corresponding alternatives. Finally, we show that the Policy Tuner can reduce the SLO violation rate by a mean of 44.6% using a single day of workload history, and that each component is efficient given its intended operating timescale.
This work outlines AutoSLO, a latency-SLO-aware work-load management framework for multi-cluster cloud data ware-houses that includes a periodic Policy Tuner that proactively plans resources using workload fore-casts, an SLO-aware reactive Autoscaler that adjusts the active cluster set based on the observed workload, and an online Query Router that reacts to concurrent query load when routing.
This work proposes and develops an open-source policy simulation framework, LoadStar, which forms a reusable benchmark pipeline for validating policies for resource-centric NoSQL workloads, and defines a resource optimization problem for placing Cosmos DB replicas onto VM nodes, and develops the Luna model for forecasting future load distributions.
Gunika Verma, Aashutosh A, Pooja Srinivas et al.· Proceedings of the VLDB Endo...· 0 citations
Modern web services are required to meet critical non-functional requirements, including availability, responsiveness, scalability, and reliability, which are formalized through Service Level Agreements (SLAs). SLAs define Service Level Objectives (SLOs), such as latency, throughput, and uptime, that ensure consistent service quality. Failing to meet these objectives can incur penalties and harm a provider's reputation. At the same time, over-provisioning resources leads to unnecessary costs and inefficient utilization. Autoscaling mechanisms address this by dynamically adjusting the number of service replicas according to demand. However, conventional approaches typically rely on low-level metrics, such as CPU or memory usage, which limit the ability to optimize both SLO compliance and infrastructure costs. This paper presents an enhanced SLO-driven autoscaling methodology for containerized workloads in Kubernetes clusters, integrating response time SLO targets into the autoscaling process. The proposed approach improves decision-making over traditional autoscaling by balancing service-level performance with operational efficiency. Experimental evaluation of a prototype demonstrates clear benefits compared to the default Kubernetes Horizontal Pod Autoscaler.
A. Marchese, O. Tomarchio· International Conference on...· 0 citations
Enterprise applications increasingly depend on distributed caching to sustain sub-millisecond response times at scale. Amazon ElastiCache Redis, operating in cluster mode, provides horizontal partitioning across configurable shard topologies, enabling throughput and memory capacity to grow in proportion to demand. While many organizations have adopted cluster configurations, empirical guidance on topology selection, key distribution optimization, and the measurable performance impact of individual tuning techniques remains sparse. This paper addresses that gap through systematic benchmarking across multiple cluster topologies (3 to 90 shards), three Graviton-based instance families (m6g, r6g, r7g), three workload profiles, and five optimization techniques, augmented by client library analysis, memory optimization guidance, and production cost validation. Production case studies from financial services, e-commerce, and real-time analytics platforms validate laboratory findings. Results offer empirical guidance for cloud architects designing caching architectures that balance latency requirements, horizontal scalability objectives, and infrastructure cost efficiency.
Kandasamy Selvaraj· International journal of com...· 0 citations
Static threshold-based autoscaling with Kubernetes Horizontal Pod Autoscaler (HPA) does not cover asynchronous event-driven workloads that only target CPU and memory usage average metrics, such as the default HPA does. The HPA targets metrics that are lagging indicators‚ unlike the queue depth and stream backlog metrics‚ and thus introduces SLO violations in document processing and data integration services․ This paper describes a production-qualified implementation for optimizing KEDA (Kubernetes Event-Driven Autoscaling) and Karpenter to manage and optimize AWS EKS pods and nodes․ KEDA extends the Kubernetes Horizontal Pod Autoscaler (HPA) for external scalers, allowing workloads' Kubernetes pods to scale based on SQS queue depth, DynamoDB Streams shard lag, CloudWatch metrics, and Prometheus time series. When combined with KEDA, Karpenter can also provision nodes from heterogeneous EC2 Spot Instance pools and consolidate nodes when underutilized. Overall, customers who have deployed Karpenter on production EKS clusters report a 40-60% reduction in compute costs and SLO adherence on variable workloads over time, as compared to EC2 On-Demand Node Groups. SLO-aware scheduling results in a 31% reduction in SLO violations and an 18% reduction in infrastructure cost. Scaling to zero (ignoring the cost of idle compute resources when there is no demand for workloads) and Spot diversification (up to 44%) optimize compute cost for off-peak workloads. Deployment patterns‚ configuration‚ graceful termination‚ and operational observability requirements are described with a focus on production deployments in enterprise environments․
Avneet Bansal· International journal of com...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
