Decentralized Hierarchical Multi-Agent DRL for Resource Allocation in IRS-Aided V2X Networks

Abstract

Vehicle-to-Everything (V2X) communication is an essential building block of intelligent transportation systems, supporting high-data-rate vehicle-to-infrastructure (V2I) services, and ultra-reliable low-latency vehicle-to-vehicle (V2V) communication. However, in dense urban environments, V2X services can be significantly degraded by the presence of severe blockage, fast channel variations, and high levels of interference. Intelligent Reflecting Surfaces (IRSs) can be employed to reconfigure wireless propagation environments to improve V2X communication. However, the joint optimization of transmit power, spectrum reuse, and IRS reflection coefficients is a mixed-integer non-linear problem, which is further complicated by the fast vehicular mobility and time-varying interference in V2X networks. To tackle this challenging problem, this work proposes a scalable and deployable decentralized hierarchical multi-agent deep reinforcement learning (DH-MDRL) framework. The key design principle is the separation of control timescales, whereby each V2V link functions as an autonomous agent that responds to local observations at a fast timescale and determines its transmit power and spectrum reuse decisions, while the IRS controller at the base station (BS), using global network observations, updates the IRS reflection coefficients at a slower timescale. This hierarchical architecture reduces coordination signaling associated with centralized resource allocation while enabling distributed resource allocation. The IRS-assisted V2X network is modeled as a Markov decision process, where the reward design is tailored to optimize the V2I sum data rate while guaranteeing the latency and reliability constraints associated with safety-critical V2V communication. Simulation results show that the proposed DH-MDRL framework outperforms conventional schemes without IRSs and achieves an excellent trade-off between V2V link constraints’ satisfaction probability and V2I link sum data rates compared to centralized resource allocation approaches.