Synergistic Dual-Loop Control for Energy Management and Voltage Stability in PV-BESS Micro-grids

Abstract

The integration of volatile renewable energy sources, such as photovoltaic (PV) systems, alongside battery energy storage systems (BESS) into DC micro-grids presents significant control challenges, primarily in ensuring DC bus voltage stability and optimizing long-term energy management. This paper introduces a novel synergistic dual-loop control architecture engineered to address these challenges. The inner control loop leverages the structural properties of differential flatness, with robust feedforward compensation, to achieve rapid and precise regulation of the DC bus voltage under disturbances. Complementing this, the outer loop employs a Q-learning-based reinforcement learning (RL) algorithm for adaptive energy resource management. The RL agent learns an optimal policy for BESS dispatch by considering real-time dynamics, including PV availability, fluctuating load profiles, battery state-of-charge (SOC), and voltage deviations, without requiring an explicit model. Validation through 24-hour simulations confirms the efficacy of the proposed architecture. The RL agent exhibits successful convergence with stable cumulative rewards. The integrated system demonstrates superior performance, maintaining tight voltage regulation with a standard deviation of 1.615 around the 120 V set-point, effective SOC management (average SOC 36.53 without limit violations), and efficient resource utilization enabling 51.49 % PV penetration. These results highlight the potential of combining nonlinear control with model-free RL to enhance hybrid micro-grid performance, resilience, and autonomy.