A hierarchical receding-horizon controller is proposed that separates slow server reconfiguration from fast throttling recourse, and offers significant speedups compared to standard optimization solvers, computing near-optimal plans within a 20 second real-time control interval.
Abstract
Cloud datacenters must increasingly modulate power in response to time-varying grid and infrastructure constraints. We study this problem as finite-horizon control of a networked hybrid dynamical system, where datacenter power and service capacity depend on interactions between servers, workers, and hosted services. Power can be reduced through fast continuous worker throttling, which acts immediately but degrades service capacity, and slow discrete server transitions, which provide deeper savings but evolve with delay. Coordinating these mechanisms yields a high-dimensional mixed-integer dynamic optimization problem which is intractable to solve at scale. We propose a hierarchical receding-horizon controller that separates slow server reconfiguration from fast throttling recourse. For fixed server states, the throttling layer reduces to a service-level convex recourse problem solved efficiently by dual decomposition. The server layer then uses a ranked-prefix search that evaluates candidate configurations through the recourse value over the planning horizon. Experiments on realistic instances with over 15,000 servers, 200,000 workers, and 1,400 services show that the controller satisfies time-varying power caps with no violations and substantially lower service impact than fast-only or slow-only baselines. Our method offers significant speedups compared to standard optimization solvers, computing near-optimal plans within a 20 second real-time control interval.
The main power modulation scenarios are reviewed, the power reduction levers using data from production cloud datacenters are characterized, Slasher's system architecture is described, and the cloud datacenter power modulation control problem is formulated.
Liuzixuan Lin, Fiodar Kazhamiaka, A. Kumbhare et al.· 1 citation
Model commitment (MC) is proposed, a mixed-integer linear programming framework that jointly schedules model deployment and cross-site request routing under power constraints and electricity-price signals and enables AI data center operators to achieve a 100% service rate under time-varying grid conditions and reduce t...
Bo-Jun Du, Hong-Yang Jia, Tong-Hui Li et al.· 0 citations
Evaluation on representative workloads demonstrates that P2CS achieves performance comparable to in-network mechanisms while significantly reducing complexity and cost, and requires minimal software changes making it readily deployable in today's datacenter infrastructure.
Ali Munir, Xiao-Lin Pang, Junyi Zhang· Conference on Applications,...· 0 citations
Janus is a Service-Engine co-designed multi-model serving system built around dual-timescale scheduling that maintains 0.97-1.0 SLO attainment, versus 0.80-0.92 for the strongest baseline, while using 13,440 device-hours/day, 27% less than static Serverless-LLM.
Tian-Bao Zhou, Yi Wang, Yu Zhou et al.· Proceedings of the ACM SIGOP...· 1 citation
RAPID is proposed, a region-aware and power-informed scheduling framework that integrates static and online heuristic schedulers for large-scale AI request scheduling that significantly reduces carbon emissions, electricity costs, and total energy consumption.
B. Ding, Cai-Ning Wang, Ka-Fei Tang et al.· Sustainability· 0 citations
PowerSlider does so with a new Flex SLO contract that turns bounded user slack into an optimization constraint, prefill--think--answer disaggregation exposing per-stage frequency and KV control, and a Karush--Kuhn--Tucker (KKT) online solver re-solving within 7.7 ms of every cap change.
Yueying Li, Jia-Yang Chen, Yuan-Fan Chen et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.