BOTVPA: An SLO-Aware and Efficient Resource Scheduling Method for Microservice

Abstract

Complex microservices topology poses significant challenges to resource scheduling. Consequently, most schedulers rely on extensive offline analysis to prevent potential SLO violations resulting from inaccurate online decisions, leading to high deployment costs. Furthermore, regarding metric selection that is important to bridge the scheduling decision to actual resource allocation, although CPU throttling-based auto-scaling has emerged as a promising approach to capture latency sensitivity, existing methods lack a quantitative mechanism to map throttling signals to resource quota, relying instead on coarse heuristics that potentially waste resources. To address these issues, we propose Bayesian Optimization and Throttling based Vertical Pod Autoscaling (BOTVPA), a sample-efficient scheduling method. BOTVPA features a two-tier architecture: 1) Central Scheduler: We employ Bayesian Optimization (BO) to enable rapid decision-making with minimal samples, eliminating offline training overhead. 2) Local Scheduler: We introduce a novel CPU throttling model that mathematically bridges the gap between throttling rates and CPU allocation. Leveraging this model, the local scheduler dynamically translates BO’s decision into precise CPU resource limits at runtime. Results show that BOTVPA reduces CPU consumption by up to 19.4% over the best-performing baseline while satisfying SLO.