2026· International journal of research and scientific innovation· Vol 13, pp. 1890-1907· 0 citations
TL;DR
The proposed DT-DRL framework establishes a closed-loop cyber–physical architecture in which continuously synchronized Digital Twin states are directly incorporated into a Proximal Policy Optimization (PPO)-based decision-making process to jointly minimize operating cost, voltage deviation, and battery degradation while satisfying network operational constraints.
Abstract
The increasing penetration of photovoltaic (PV) generation and battery energy storage systems (BESSs) has significantly increased the operational complexity of active distribution networks, where real-time energy management must simultaneously address renewable uncertainty, voltage regulation, and battery lifetime preservation. Existing Digital Twin-based energy management approaches primarily support monitoring and visualization, whereas deep reinforcement learning (DRL) controllers are commonly developed independently of real-time system synchronization, limiting their adaptability under rapidly changing operating conditions. To overcome these limitations, this paper proposes a Digital Twin-enabled Deep Reinforcement Learning (DT-DRL) framework for coordinated PV–BESS energy management in active distribution networks. The proposed framework establishes a closed-loop cyber–physical architecture in which continuously synchronized Digital Twin states are directly incorporated into a Proximal Policy Optimization (PPO)-based decision-making process. A multi-objective formulation is developed to jointly minimize operating cost, voltage deviation, and battery degradation while satisfying network operational constraints. Renewable generation and load uncertainties are represented using Monte Carlo-based stochastic scenarios to improve policy robustness under practical operating conditions. The proposed framework is validated on the IEEE 33-bus distribution system and compared with rule-based control (RBC), optimal power flow (OPF), and conventional DRL approaches. Simulation results demonstrate that the proposed method reduces the daily operating cost by 22.2%, decreases the maximum voltage deviation to 0.039 p.u., and achieves more stable BESS operation with lower operational variability under uncertain conditions. Furthermore, the complete Digital Twin synchronization and PPO decision-making process requires only 0.41 s per control interval, satisfying the timing requirements of distribution-level energy management systems. These results demonstrate that the proposed DT-DRL framework provides an accurate, computationally efficient, and practically deployable solution for real-time energy management in renewable-rich active distribution networks.
The integration of volatile renewable energy sources, such as photovoltaic (PV) systems, alongside battery energy storage systems (BESS) into DC micro-grids presents significant control challenges, primarily in ensuring DC bus voltage stability and optimizing long-term energy management. This paper introduces a novel synergistic dual-loop control architecture engineered to address these challenges. The inner control loop leverages the structural properties of differential flatness, with robust feedforward compensation, to achieve rapid and precise regulation of the DC bus voltage under disturbances. Complementing this, the outer loop employs a Q-learning-based reinforcement learning (RL) algorithm for adaptive energy resource management. The RL agent learns an optimal policy for BESS dispatch by considering real-time dynamics, including PV availability, fluctuating load profiles, battery state-of-charge (SOC), and voltage deviations, without requiring an explicit model. Validation through 24-hour simulations confirms the efficacy of the proposed architecture. The RL agent exhibits successful convergence with stable cumulative rewards. The integrated system demonstrates superior performance, maintaining tight voltage regulation with a standard deviation of 1.615 around the 120 V set-point, effective SOC management (average SOC 36.53 without limit violations), and efficient resource utilization enabling 51.49 % PV penetration. These results highlight the potential of combining nonlinear control with model-free RL to enhance hybrid micro-grid performance, resilience, and autonomy.
H. Chabana, I. Tegani, Salem Tegani et al.· Electrotehnică, electronică,...· 0 citations
This paper proposes a dual-layer coordinated framework that combines day-ahead battery energy storage system (BESS) scheduling with real-time Volt–VAr Control (VVC) for active distribution networks. The optimization minimizes distribution system technical losses while satisfying operational constraints related to voltage regulation, equipment loading, battery operation, voltage regulator (VR) tap commutation, and smart inverter (SI) operating limits defined by IEEE Std 1547-2018. The planning stage determines the optimal charging and discharging schedule of multiple BESS units over a 24-hour horizon, whereas the operational stage performs real-time VVC through the coordinated control of VRs, capacitor banks (CBs), and SI associated with distributed photovoltaic (DPV) and BESS units. The methodology was implemented in a Python–OpenDSS co-simulation environment and validated on a modified IEEE 34-bus feeder using real SCADA load measurements and solar irradiance data through daily and seasonal operating scenarios under both planning and actual operating conditions. Performance was also compared with conventional local VVC strategies. Results demonstrate that the proposed framework maintains voltages within prescribed limits, eliminates or substantially mitigates reverse power flow, reduces feeder peak demand, and significantly decreases network energy losses. Overall, the proposed strategy significantly enhances the operation of active distribution networks with high renewable energy penetration.
R. R. Biazzi, D. Bernardon, Maurício Sperandio· IEEE Access· 0 citations
The rapid integration of renewable energy resources into modern distribution networks has significantly improved the
sustainability of electrical power systems. However, the intermittent characteristics of photovoltaic (PV) and wind energy
conversion systems (WECS), together with the increasing penetration of nonlinear loads, have introduced serious power quality
challenges, including harmonic distortion, voltage fluctuations, reactive power imbalance, and poor dynamic stability. This
paper proposes an intelligent hybrid microgrid incorporating a Photovoltaic (PV) system, Wind Energy Conversion System
(WECS), Battery Energy Storage System (BESS), and Unified Power Quality Conditioner (UPQC) controlled by an Adaptive
Neuro-Fuzzy Inference System (ANFIS). The proposed ANFIS controller replaces the conventional proportional-integral (PI)
controller to achieve faster dynamic response, adaptive reference current generation, and superior DC-link voltage regulation
under varying operating conditions. The coordinated operation of PV, WECS, and BESS ensures continuous energy availability
while minimizing dependence on the utility grid. The UPQC effectively compensates voltage disturbances, mitigates harmonic
currents, and maintains near-unity power factor, thereby improving the overall power quality of the hybrid microgrid.
MATLAB/Simulink simulations demonstrate that the proposed ANFIS-based strategy significantly reduces total harmonic
distortion (THD), enhances transient performance, improves renewable energy utilization, and increases system reliability
compared with the conventional PI-controlled configuration, making it an effective solution for future smart grid applications
D. Sangeetha, Kadingu Shivashankar· International Journal for Re...· 0 citations
The presence of intermittent sources of renewable energy in power systems requires ESSs to manage temporal imbalance in energy supply and demand. In this study, we introduce a hybrid energy storage system (HESS) coupled with an AI energy management system (EMS) that uses deep reinforcement learning (DRL) for optimal scheduling of renewable energy utilization within grid-connected and islanded microgrids. AI-enabled EMS utilizes a DRL agent with proximal policy optimization (PPO) to make optimal decisions regarding energy generation based on state space and economic considerations, while accounting for SoC constraints of batteries. An important aspect of the proposed system is the design of HESS architecture and reward function based on DRL. Further improvements are made via analysing the PPO clipping sensitivity, Pearson correlation analysis on the relationship between the intermittency of renewables and response latency, and Monte Carlo uncertainty analysis with a 95% confidence interval. For a 24-hour simulation period, the developed system is able to cut down on grid power imports by 43.2%, have an 87.3% renewable energy utilization rate, extend the lifespan of the battery from 8.1 to 12.5 years, and have 91.4% peak shaving efficiency through 100 Monte Carlo runs and without any SoC violations (25%–90%). Net benefit analysis is estimated to be $56,000–$66,000 for 15 years at a 6% discount rate, while a 120 ms response time and one-way ANOVA with Tukey's HSD confirm statistical significance.
Lalit Sachdeva, U. Anand· Energy Storage and Conversio...· 0 citations
The increasing integration of renewable energy sources into microgrids has intensified the need for intelligent energy management strategies capable of addressing the intermittency of solar and wind generation while ensuring reliable and cost-effective operation. Although Rule-Based Control (RBC) methods are straightforward to implement, their limited adaptability often leads to suboptimal utilisation of Hybrid Energy Storage Systems (HESS). This study develops and evaluates a Deep Reinforcement Learning (DRL)-based energy management system employing a Deep Q-Network (DQN) to coordinate battery–supercapacitor operation within a renewable microgrid. A Gymnasium-compatible simulation environment was constructed using a publicly available time-series dataset comprising renewable generation, load demand, electricity prices, battery state of charge (SoC), and supercapacitor SoC. Feature engineering, incorporating sinusoidal temporal representations and Min-Max normalisation, was applied to enhance learning stability and capture cyclical demand and generation patterns. The DQN agent was trained over 50 episodes and benchmarked against a conventional RBC strategy under identical operating conditions. Training performance demonstrated progressive policy improvement, with cumulative rewards increasing from approximately -1200 to -400, indicating enhanced decision-making capability during learning. The learned controller exhibited adaptive energy scheduling through dynamic utilisation of the supercapacitor and selective grid interaction in response to varying operating conditions, whereas the RBC followed a deterministic control strategy with limited flexibility. However, comparative evaluation revealed that the DQN did not consistently outperform the RBC in cumulative economic performance, suggesting the need for further refinement of the reward function, training process, and hyperparameter configuration. Nevertheless, the proposed framework demonstrates the feasibility of applying deep reinforcement learning to coordinated battery–supercapacitor energy management and highlights its potential to enhance operational flexibility and intelligent resource utilisation in renewable microgrids. The study contributes a dataset-driven reinforcement learning framework that provides a foundation for future research on advanced AI-based energy management systems and the integration of more sophisticated reinforcement learning algorithms for resilient and sustainable microgrid operation.
Daniel Owusu· American Journal of Neural N...· 0 citations
Growing deployment of distributed renewable energy resources is essential to realizing sustainability and carbon neutrality in contemporary power networks. Nevertheless, the increasing penetration of these resources in power distribution networks can create operational uncertainties due to solar intermittency, stochastic load variations, and voltage instability. Thus, efficient coordination of these dispersed resources needs smart energy management schedules that have the capability to adapt dynamically under varying operational situations. This work proposes a PV–battery integrated 82-bus Owerri urban distribution network model using MATLAB/Simulink. To address the real-time uncertainties in solar generation and load demand, an AI– based energy management framework is proposed. This is done through a framework that integrates RL with PSO to dynamically cooperate with photovoltaic generation, battery energy storage, and grid power. BFS methodology is adopted for the computation of power flow applied to the distribution grid under stochastic operating conditions for the evaluation of voltage stability and power loss performance. Numerical results showed that the proposed AI-based energy management system is superior to traditional manual control strategies. It was equally observed that the RL-based controller enhanced the voltage magnitude from 0.86 to 0.96 p.u. while the value of real power loss declined by 42.9% (210kW-120kW). The findings validated that the proposed adaptive AI-based energy management can significantly increase the incorporation of PV systems, voltage stability, and operational resilience for forthcoming smart network distribution systems, in line with standards stipulated by IEEE, NERC, and NEMSA.
Okwe Gerald, Mustapha Abdullahi, Okafor Izuchukwu et al.· International journal of rec...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
