Strategic flight planning and conflict management for urban air mobility operations: a mission preference-constrained multiagent reinforcement learning approach

Abstract

As an emerging low-altitude transportation paradigm, urban air mobility (UAM) is envisioned to support high-density and demand-driven operations involving diverse and flexible mission requests. However, the imbalance between limited urban airspace resources and growing operational demands inevitably causes frequent flight conflicts, posing significant challenges to safe and efficient operations. To address this issue, this study proposes a multiagent reinforcement learning (MARL) approach to achieve strategic four-dimensional trajectory (4DT) flight planning and conflict management during the preflight window. First, a collaborative optimization framework is established, in which the deconfliction problem is formulated as a multiagent Markov decision process (MAMDP) to enable coordinated decision-making. Then, a mission preference-constrained MARL method is developed by integrating two specialized mechanisms into the multiagent deep deterministic policy gradient (MADDPG) algorithm to address UAM operational characteristics. Specifically, an action masking for mission preference (AMMP) mechanism is implemented to ensure execution compliance, and a hierarchical prioritized experience replay (HPER) mechanism is designed to improve learning efficiency. Simulation results demonstrate that the proposed AMMP-HPER-MADDPG (AH-MADDPG) method achieves an average conflict resolution rate of nearly 96% and a preference awareness rate of 100% in scenarios involving 100 flight plans, significantly outperforming other methods. The proposed approach provides an effective and adaptive solution for ensuring operational safety, mission preference, and flight efficiency in future UAM operations.