Hierarchical cooperative signal control for high-demand networks within the MFD framework: A deep reinforcement learning approach

Abstract

Owing to its powerful modeling and decision-making capabilities in complex urban networks, deep reinforcement learning (DRL) has garnered significant attention in both regional signal control and perimeter control. However, regional control tends to fail under oversaturated conditions, while perimeter control often leads to excessive release due to its limited perception of internal traffic states. Although existing studies have attempted to combine these two strategies to overcome the limitations of single control approaches, developing an efficient information exchange and coordination mechanism between them remains challenging. To address this issue, this paper proposes a Regional Hierarchical Cooperative Control (RHCC) framework based on DRL. The upper layer performs regional control to optimize internal signal operations and mitigate local congestion, whereas the lower layer applies perimeter control guided by the Macroscopic Fundamental Diagram (MFD) to regulate inflow and outflow, thus preventing oversaturation. The two layers work cooperatively to achieve global traffic signal optimization. Furthermore, a multi-level communication and coordination mechanism is designed to establish a shared knowledge base, enabling efficient information exchange among homogeneous and heterogeneous agents across layers and alleviating information deficiency under partially observable environments. Finally, considering the spatial and functional differences between upper and lower level agents, We designed an improved PinSAGE algorithm to enhance environmental perception, feature aggregation, and decision-making capabilities across different layers. Experimental results demonstrate that the proposed RHCC framework maintains superior control performance even under high-demand traffic conditions and outperforms existing state-of-the-art methods in terms of both macroscopic efficiency and microscopic throughput.