Jul 2026· Applied Sciences· Vol 16, pp. 6685· 0 citations
TL;DR
The proposed Agent SAC-Reinforcement Learning control was tested on a two-area microgrid cluster, which demonstrated greater frequency stability and a frequency nadir deviation improvement and ROCOF reduction compared to PID-based strategies.
Abstract
Traditional control strategies, such as droop-frequency and PI controllers, often show limited adaptability when the system operates under highly variable renewable generation and load conditions. In order to overcome this limitation, this study proposes the design and simulation of a clustered microgrid supported by reinforcement learning (RL)-based virtual inertia control. Three continuous-control RL algorithms were evaluated: Deep Deterministic Policy Gradient (DDPG), Twin-Delayed Deep Deterministic Policy Gradient (TD3), and Soft Actor-Critic (SAC). The SAC agent provided the most robust training performance, reaching stable convergence after approximately 400 episodes and a final reward close to 86 units after 600 episodes. DDPG presented the second-best behavior, whereas TD3 achieved the lowest final reward, approximately 43 units. The proposed Agent SAC-Reinforcement Learning control was tested on a two-area microgrid cluster, which demonstrated greater frequency stability. Results indicate a frequency nadir of 59.82 Hz and ROCOF of 0.1484 Hz/s, with 6.78% nadir deviation improvement and 37.23% ROCOF reduction compared to PID-based strategies.
To improve the ride comfort and attitude stability of the vehicle under complex driving conditions, this paper proposes a distributed model predictive control (DMPC) strategy with an adaptive weight-tuning mechanism based on the deep deterministic policy gradient (DDPG) algorithm for the active suspension system. The proposed method addresses the strong coupling among body vertical, pitch, and roll vibration-control objectives. It also reduces the reliance of conventional controllers on empirical parameter tuning and improves their adaptability to varying conditions. This study establishes a seven-degree-of-freedom full-vehicle active suspension model and decomposes it into a body subsystem and four wheel subsystems according to the coupling relationships. Then, a distributed predictive control framework is constructed. In this framework, local receding-horizon optimization and limited information exchange are used to achieve coordinated control. Furthermore, the DDPG algorithm learns the dynamic characteristics of the system online and adaptively adjusts the weighting parameters of the DMPC controller in real time. This enables dynamic allocation of control effort under varying operating conditions. The simulation results obtained from a high-fidelity CarSim co-simulation platform show that the proposed method effectively suppresses body vertical, pitch, and roll vibrations under different operating conditions. In addition, the proposed strategy reduces the average computation time compared with conventional MPC. Hardware-in-the-loop experiments further validate the effectiveness and real-time performance of the proposed controller.
Huichao Zhang, Jiayu Lu, Bo Li et al.· Journal of Vibration and Con...· 0 citations
The increasing penetration of inverter-based renewable generation has reduced system inertia and posed new challenges to frequency stability in modern power systems. Virtual synchronous generator (VSG) control can provide virtual inertia and damping support, but its performance strongly depends on the proper coordination of these parameters under varying operating conditions. Existing adaptive and reinforcement-learning-based methods usually regulate virtual inertia and damping at the same timescale, which may ignore their distinct physical roles and lead to coupled parameter variations. To address this issue, this paper proposes a bi-timescale hierarchical safe reinforcement learning framework, termed BiTS-HSRL-JD, for coordinated virtual inertia and damping control in grid-forming converters. In the proposed framework, a slow-timescale policy schedules virtual inertia according to operating conditions, while a fast-timescale policy adjusts the damping coefficient to suppress transient oscillations. A stage-aware state representation and a safety projection layer are further introduced to improve transient adaptability and enforce practical constraints on parameter bounds and variation rates. The proposed method is validated using a MATLAB/Simulink-based VSG system under strong-grid and weak-grid conditions, power-step disturbances, and load-switching events. Comparative results show that BiTS-HSRL-JD reduces RoCoF, improves frequency recovery, and suppresses oscillations more effectively than fixed-parameter, rule-based adaptive, and single-policy reinforcement learning methods.
Zhilin Dong, Haoqing Xiong, Rongqian Su et al.· IEEE Access· 0 citations
The increasing penetration of renewable energy sources (RES) and plug-in hybrid electric vehicles (PHEVs) has introduced significant frequency instability in interconnected microgrid (MG) systems, necessitating adaptive and robust control strategies for reliable operation. This paper proposes a fuzzy-explainable neural network (FxNN)-based distributed fractional-order PID (FOPID) controller for active frequency regulation in interconnected microgrids. The control problem is formulated within a Lyapunov-based optimization framework, where system stability is ensured through a recursively updated energy function and differential learning dynamics of the explainable neural network. The proposed cascaded FxNN-FOPID controller is evaluated under varying load disturbances and intermittent renewable power. Simulation results demonstrate superior dynamic performance compared with conventional single-loop FxNN and PSO-GSA-based controllers. The proposed controller achieves a minimum settling time of 3.0 s, representing improvements of 14.3% and 30.2%, respectively. Furthermore, the integral absolute error (IAE) is reduced to 0.6292 × 10
−3
, 0.5929 × 10
−3
, and 0.6000 × 10
−3
for MG1, MG2, and MG3, respectively, while the integral time absolute error (ITAE) is reduced by up to 93.6%. The controller also minimizes frequency oscillations with a peak magnitude of 0.0003 p.u. and achieves improved Integral of Squared Error (ISE) 0.1040 × 10
−5
and Integral of Time-weighted Squared Error (ITSE) 0.9918 × 10
−3
values. Owing to its adaptive gain tuning capability, the proposed controller maintains robust performance without requiring manual retuning under varying operating conditions. Comparative analysis confirms that the proposed cascaded FxNN-based distributed FOPID controller provides faster dynamic response, improved disturbance rejection, and superior low-frequency oscillation damping, making it an effective solution for reliable frequency regulation in renewable-energy-integrated microgrid systems. Furthermore, the proposed framework supports Sustainable Development Goals (SDG 7: Affordable and Clean Energy, SDG 9: Industry, Innovation and Infrastructure, and SDG 13: Climate Action) by facilitating resilient microgrid operation, enhancing renewable energy integration, and promoting low-carbon power systems.
This research introduces an innovative control technique for Series Elastic Actuators (SEAs) that utilizes Reinforcement Learning (RL) to address the shortcomings of previously fixed-gain adaptive controllers, which are a hybrid of State Feedback Control (SFC) and Model Reference Adaptive Control (MRAC) by using Lyapunov Stability Analysis. This controller is optimized by adjusting the adaptation factor. b. This study presents an intelligent agent based on reinforcement learning to find the value of b with a dynamic auto-tuner. It trains via the Soft Actor-Critic (SAC) algorithm for 100,000 time steps. A comparison between the two methods was presented according to simulation results under different conditions; the RL-based controller shows much better tracking accuracy, how quickly it reaches the target output, and how little control torque it uses, where the agent's policy could automatically adjust in real-time based on system conditions, such as uncertainties and disturbances, where it has a settling time of 1.7 seconds, while the fixed parameter controller has a 1.95-second settling time, resulting in a reduction of 15.3%. It also lowers the control torque caused by disturbances by 19.5% compared to the fixed parameter controller, which has a control torque of 3.99 Nm, while the maximum control torque for the RL-optimized controller is 3.21 Nm.
H. Z. Abdalikhwa, Waleed Al-Ashtari· International journal of com...· 0 citations
Microgrids play a critical role in enhancing the flexibility, reliability, and sustainability of modern power systems by integrating distributed energy resources, energy storage systems, and controllable loads. However, the inherent uncertainty of renewable generation and the stochastic nature of load demand pose significant challenges to optimal energy management. To address these issues, this paper proposes a deep reinforcement learning (DRL)-based optimal energy management framework for microgrids. The problem is formulated as a Markov decision process, where the system state captures renewable generation, load demand, and storage status, while the control actions determine power dispatch and energy storage operation. A deep reinforcement learning model is developed to learn optimal control policies through continuous interaction with the environment, enabling adaptive decision-making under dynamic and uncertain conditions. To improve learning efficiency and policy stability, state normalization and reward shaping strategies are incorporated. Furthermore, a constrained optimization mechanism is introduced to ensure operational safety and economic feasibility. Experimental results on benchmark microgrid scenarios demonstrate that the proposed method outperforms conventional rule-based strategies and model-based optimization approaches in terms of operational cost reduction, energy utilization efficiency, and robustness under uncertainty. The results indicate that the proposed DRL-based framework provides an effective and scalable solution for intelligent microgrid energy management.
Li Chen, Hong-Qiao Li, Zhen-Xing Chen et al.· European Conference on Elect...· 0 citations
Convergence and multi-run statistical analysis further confirm the robustness, stability, and reproducibility of the trained policy, demonstrating the effectiveness of DRL as an intelligent and scalable solution for next-generation microgrid PQ control.
Pratibha V. Hurkadli, G. A. Kumar, T. Manjunath· Advances in Data Science and...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
