Pre-control scheme generation for extreme-weather power imbalance risk using multi-resource frequency regulation and deep reinforcement learning

Abstract

To address the power imbalance risk between renewable energy output and load demand under extreme weather conditions, this paper proposes a pre-control scheme generation method based on the integration of multiple frequency regulation resources and deep reinforcement learning. First, mechanism models for wind power and photovoltaic output, along with an hourly time-series energy storage model, are established to quantify the supply-demand imbalance risk under extreme weather scenarios. Second, a reward function embedded with active regulation flexibility physical rules is designed and incorporated into an improved deep deterministic policy gradient (DDPG) algorithm framework. During the policy update process, equipment operating boundary constraint penalties and flexibility incentives are introduced, and the physical consistency of the policy is enhanced through dynamic constraint construction, adaptive learning rate adjustment, and policy visualization. Third, based on the DDPG framework, an entropy-regularized twin-delayed network algorithm is incorporated, which employs a dynamic entropy term to enhance exploration capability and utilizes twin-delayed networks to mitigate overestimation of the value function. Finally, experimental results demonstrate that the proposed method achieves a total cost of 120.6 CNY, a success rate of 94.7%, and a violation rate of 0.0% under extreme weather scenarios, all outperforming the comparative methods. Ablation experiments validate the synergistic contribution of each improved module, and visualization results further confirm the temporal rationality and boundary constraint compliance of the generated pre-control schemes.