Deep reinforcement learning-based adaptive FOPID tuning for power system stability enhancement: A twin delayed deep deterministic policy gradient approach

Abstract

Abstract This paper proposes a Twin Delayed Deep Deterministic Policy Gradient (TD3)-based adaptive fractional-order proportional-integral-derivative (FOPID) controller for joint load-frequency and voltage regulation in a renewable-rich power system. The proposed controller adaptively tunes five FOPID parameters, namely $${K_p}$$ , $${K_i}$$ , $${K_d}$$ ,λ, and µ, through a multi-objective reward formulation that simultaneously minimizes frequency integral time-weighted absolute error (ITAE), terminal-voltage ITAE, maximum frequency deviation, and tie-line power error. A curriculum-based training strategy is employed to progressively expose the TD3 agent to increasing disturbance severity. The proposed framework is evaluated on a modified IEEE 14-bus system incorporating 46% renewable penetration, comprising DFIG-based wind generation and photovoltaic generation under low-inertia operating conditions. Under the standard 0.10 p.u. step-load disturbance, the proposed TD3-FOPID controller achieves an ITAE of 0.0612, a settling time of 1.48 s, and an overshoot of 1.82%. Across four nominal disturbance scenarios, the obtained damping ratios range from 0.74 to 0.78, while the broader tested operating-envelope analysis gives a minimum damping ratio of 0.72. The maximum system-level ROCOF observed at disturbance inception across the evaluated scenarios is 0.82 Hz/s, below the 2 Hz/s reference threshold adopted in this study. Statistical evaluation over ten independent training/evaluation runs with different random seeds further demonstrates consistent performance of the proposed approach. Numerical Lyapunov verification using the linearized closed-loop model provides numerical evidence of stable closed-loop behaviour within the tested operating envelope. The results demonstrate the potential of TD3-based five-parameter FOPID adaptation for robust frequency and voltage regulation in renewable-rich power systems.