Local Frame-Enhanced Deep Reinforcement Learning for Multiagent Pursuit-Evasion Game

Abstract

The multiagent pursuit-evasion game (PEG) with obstacle avoidance remains a challenging cooperative-competitive problem. A key difficulty is achieving effective cooperation and obstacle avoidance in nonstationary environments. Previous studies typically assume that agents can automatically obtain accurate obstacle position information within a certain range, while overlooking the role of sensors in environmental perception. In this article, we use deep reinforcement learning to investigate a scenario in which multiple pursuers capture a single evader in a partially observable environment. Instead of providing agents with the precise locations of obstacles and teammates, we equip them with 2D-LiDAR sensors to perceive their surroundings. To address the limited environmental information in a single frame of 2D-LiDAR data, we introduce a spatiotemporal attention network with physics-informed positional information to process temporal LiDAR sequences. Based on this design, we propose a QMIX-based cooperative collision-free capture policy. Experimental results demonstrate that the proposed method enables pursuers to avoid collisions effectively and outperforms other baseline methods. To the best of our knowledge, existing PEG studies have rarely conducted systematic robustness evaluation under varying obstacle densities and multiple evader policies. In this work, we further analyze the impact of different obstacle densities and evader policies on the algorithm’s performance, and the results show that our method exhibits strong generalization ability.