Cooperative control of multi-microgrids under uncertainty: a multi-agent actor-critic approach with LSTM and deep dense architecture

Abstract

The high penetration of renewable energy poses significant source-load uncertainty challenges to the economic and stable operation of microgrids. To address the key challenges of multi-microgrid cooperative optimization, including temporal source-load uncertainty, heterogeneous inter-microgrid coordination, and real-time decentralized decision-making, this paper proposes a cooperative control strategy named Long Short-Term Memory-based Multi-Agent Actor-Critic (L-MAAC). Under the Centralized Training with Decentralized Execution (CTDE) framework, the proposed method learns cooperative policies using global system information during training, while enabling each microgrid to make decisions based only on local observations during execution. Specifically, the Actor network incorporates an LSTM module to extract temporal features from historical source-load data, thereby improving the ability to proactively respond to renewable generation and load fluctuations, while the Critic network integrates an individual attention mechanism with a Deep Dense architecture in Reinforcement Learning (D2RL), which enhances the representation of heterogeneous interactions among microgrids and alleviates information loss in deep value-function approximation. Ablation studies, reward sensitivity, and hyperparameter sensitivity analyses confirm the synergistic contributions of these components. Simulation results based on a modified IEEE 13-bus multi-microgrid system show that, compared with advanced algorithms such as MAAC, L-MAAC reduces the total system operating cost by 4.81% on the test set. Furthermore, under multiple uncertainty scenarios, including cloud dynamics, wind power randomness, sudden load changes, and combined electricity price/diesel generator failures, the proposed method demonstrates superior voltage regulation capability and robustness. These results indicate that L-MAAC can effectively improve both the economic efficiency and operational stability of multi-microgrid systems under uncertain environments.