Optimization of dynamic admission control for preemptible cloud services based on TPD-SAC reinforcement learning

Abstract

Abstract Preemptible cloud services utilize idle resources from contract-based services. The resources allocated to these low-priority services can be preempted by contract-based services during periods of overload, so cloud platforms offer low prices to attract cost-sensitive users. While preemptible cloud services enhance resource utilization and boost revenue, excessive admission of service requests may induce overbooking effects, resulting in an excessively high preemption rate that undermines user satisfaction and ultimately lowers revenue. Conversely, insufficient admissions may result in underutilized resources. The stochastic nature of idle resource capacities necessitates real-time dynamic optimization of admission control. The primary difficulty in optimizing dynamic admission control lies in the fact that fluctuations in idle resource capacity are a special category of stochastic fluctuations characterized by non-stationarity, where the stochastic distribution patterns evolve over time. To address this difficulty, we first formulate the admission control for preemptible cloud services with overbooking effects as a Non-Stationary Markov Decision Process (NSMDP). Then, an improved soft actor-critic (SAC) framework is proposed for the NSMDP model. In this framework, the regularization and feature-enhanced LSTM algorithm is employed for turning point detection (TPD) in non-stationary stochastic fluctuation patterns. On this basis, a soft actor-critic algorithm based on turning point detection (TPD-SAC) is developed to achieve real-time resolution of NSMDP. Experiments with Google Cloud datasets validate the effectiveness of the proposed algorithm in non-stationary environments. Furthermore, simulation experiments were conducted to analyze the revenue contributions of dynamic admission strategies under two pricing mechanisms. Experimental results indicate that compared to strategies under the uniform discount mechanism, dynamic admission strategies achieve higher revenue contributions under the interruption-based discount mechanism.