Multi-Agent Reinforcement Learning for Conflict Resolution in Air Traffic Control With Delayed Actions
The aim of air traffic control is to guide aircraft from its departure to its destination in a safe and efficient manner. Keeping the aircraft separated in accordance with the regulations is one of the most crucial tasks, which in dense traffic often induces a high workload for the air traffic controller. Automated support systems for conflict detection and resolution would be a valuable addition to the air traffic control systems and has been an active area of research for decades. Reinforcement learning has recently been suggested as a promising alternative to traditional algorithms. A key challenge in air traffic control is that there is a significant delay between an action and its execution, which can have a significant negative impact in reinforcement learning. In this paper we show that by adding data from the trajectory prediction system to the observation space in combination with larger networks and reward sharing, we can overcome the problems introduced by the action delays. We also present a novel method of creating a low-cost ensemble by combining the agents from one training session and using the critics value function to vote on the actions to further improve our results.