Results suggest that HG-MARL is a promising simulation-validated framework for civilian UAV swarm coordination in collision-and-separation-critical and communication-degraded environments.
Abstract
Safe-separation-and-collision-avoidance unmanned aerial vehicle (UAV) swarms are increasingly used for inspection, emergency response, environmental monitoring, and search-and-rescue support in cluttered airspace where communication links may be delayed, degraded, or intermittently unavailable. These applications require heterogeneous vehicles to maintain situational awareness, allocate tasks, and avoid hazards under partial observability and changing team topology. To address these challenges, this paper proposes a Hierarchical Graph-Attention Multi-Agent Reinforcement Learning architecture (HG-MARL) for safe-separation-and-collision-avoidance heterogeneous UAV swarm coordination. The proposed framework decomposes the task into high-level resource allocation and low-level local-control execution, uses graph attention for changing swarm topology, and applies Transformer memory, action masking, potential-field reward shaping, and domain-randomized simulation training. In the multi-scenario simulation summaries, HG-MARL achieves 92.9%, 89.8%, and 82.6% task success in Scenarios A–C, respectively, improving upon MAPPO by 15.1, 21.4, and 20.1 percentage points. Summary-statistic Welch tests show that all six HG-MARL comparisons against MAPPO and QMIX yield p<0.01 with large effect sizes. Fair-control, reward-sensitivity, communication-degradation, safety-ablation, training-stability, latency, and transfer-oriented stress tests further support the contributions of the integrated architecture. The validation scope is simulator-based, with platform-level flight/HIL evaluation discussed as future work. These results suggest that HG-MARL is a promising simulation-validated framework for civilian UAV swarm coordination in collision-and-separation-critical and communication-degraded environments.
Coordinating multiple unmanned aerial vehicles (UAVs) for inspection, delivery, and search-and-rescue requires routing strategies that are both efficient and safety-aware under partial observability. Conventional optimisation methods and single-level reinforcement-learning policies often struggle when fleet size and obstacle density increase, since local collision avoidance and long-horizon route coordination must be learned within the same primitive-action policy. We reformulate multi-UAV path planning as a hierarchical reinforcement-learning problem and propose a two-tier controller in which a recurrent PPO+LSTM Manager selects coarse waypoint-level decisions, while a shared recurrent PPO+LSTM Worker executes local collision-aware movements. The Worker is pre-trained and then frozen, allowing the recurrent Manager to learn macro-level coordination over a reduced decision horizon. Under idealised deterministic grid assumptions, we show that enlarging the waypoint dictionary weakly expands the feasible macro-action set and can preserve safe alternatives when such alternatives exist. Experiments on controlled \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$10\times 10$$\end{document} grid scenarios with 2–6 UAVs and 20–40 % obstacle coverage show that hierarchical PPO+LSTM achieves 90–93 % mission success and collision rates of at most 10 %, outperforming flat PPO and PPO+LSTM baselines. The evaluation further includes reward-coefficient sensitivity analysis, high-resolution path-planning visualisations, a hard bottleneck feasibility illustration, MAPPO-style diagnostic convergence curves, and rollout-loop runtime diagnostics. Statistical validation using Welch’s t-tests supports the significance of the main performance improvements. The results indicate that separating waypoint-level strategy from recurrent local execution improves mission reliability and collision avoidance in the tested grid environments, while larger random-map benchmarks, fully controlled MAPPO/QMIX comparisons, and continuous 3-D simulation remain important future work.
Safe cooperative navigation of unmanned aerial vehicle (UAV) formations through three-dimensional environments with dense obstacles, dynamic threats, and unstructured terrain requires jointly addressing goal-directed navigation, formation keeping, hard safety constraints, and formation-topology consistency. We present FALCON-MASAC, a safety-integrated multi-agent reinforcement learning framework that decomposes this task into four complementary layers: (1) a hierarchical leader-follower paradigm that pairs a pre-trained virtual leader with followers learning a distributed cooperative policy; (2) a dual-scale entity-risk attention encoder (DSER-AE) that structures heterogeneous observations into entity-scale and risk-scale semantic tokens and fuses them through intra-scale self-attention and inter-scale full-sequence self-attention over the concatenated tokens; (3) a safety shield built on signed distance functions and high-order control barrier functions (SDF-HOCBF) that maintains the conditional safety certificate while the robust quadratic program remains feasible and the high-order admissibility conditions hold, and switches to a bounded best-effort fallback otherwise; and (4) a bypass-side commitment coordination layer that suppresses trajectory chattering and mitigates crossing conflicts among neighboring UAVs. The safety analysis explicitly quantifies the SDF linearization error and dynamic-obstacle prediction uncertainty and describes the bounded fallback used when certified execution is unavailable. Under the centralized-training-with-decentralized-execution (CTDE) paradigm, simulation experiments show that FALCON-MASAC substantially outperforms representative baselines: it attains a 97.3% success rate (21.7 percentage points above the strongest pure-MARL baseline and 13.0 percentage points above a post-hoc CBF-RL variant), a success-conditional steady-tracking formation error of 1.524 m, and a minimum clearance of 2.732 m. The framework also generalizes well along two out-of-distribution dimensions: obstacle density and dynamic-obstacle speed.
Yiming Shang, Changping Du, Rui Yang et al.· Unmanned Systems· 0 citations
T-CARE is introduced, a hybrid learning-heuristic multi-UAV coordination framework that integrates zero-shot constrained action selection with priority-aware temporal reservations and achieves 100% success, 0% collision rate, and no observed persistent starvation or deadlock.
A multi-agent deep reinforcement learning framework that addresses issues through coordinated exploration, demonstration exploitation, safe curriculum scheduling, and structure-aware generalisation is proposed, demonstrating strong performance in collaboration success rate, navigation robustness, zero-shot cross-scenario generalisation, and dynamic environment adaptability.
The proposed framework demonstrates robust scalability and real-time coordination capability for dynamic environments, while providing a reliable decision-making paradigm for intelligent multi-agent systems operating in communication-intensive and electromagnetically complex application scenarios.
X.-H. Fang, K. Chen, Cheng-Hao Ren et al.· Advanced Electromagnetics· 0 citations
An Attention-based Multi-Agent Deep Deterministic Policy Gradient algorithm was developed for cooperative multi-unmanned aerial vehicle target tracking in dynamic environments. The study addressed information redundancy and association weight allocation between individual unmanned aerial vehicles and the swarm during cooperative decision-making. To improve information selection, the proposed algorithm introduced a centralized critic network with a multi-head attention mechanism to evaluate the contributions of other agents at each time step. Meanwhile, the study designed a dual-buffer experience management architecture composed of a recent interaction memory and a mission outcome memory. This architecture stored recent interaction data and mission-critical trajectories separately, thereby improving experience utilization during training. The study also formulated the state space, action space, and reward function for target exploration, obstacle avoidance, energy consumption, and velocity maintenance under limited perception conditions. The experiments evaluated the proposed algorithm in a two-dimensional tracking scenario with moving targets, multiple unmanned aerial vehicles, and dynamic and static obstacles. The proposed method was compared with Deep Deterministic Policy Gradient and Multi-Agent Deep Deterministic Policy Gradient using collision rate, capture rate, capture time, and capture distance as evaluation metrics. The results showed that the proposed algorithm improved convergence behavior and average reward, reduced the collision rate by 83% compared with Deep Deterministic Policy Gradient at the first environmental level, and maintained competitive performance in capture rate, capture time, and path efficiency.
Qinglin Han, Hongmei Wang· International Conference on...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
