Aug 2026· International Conference on Circuit, Power and Computing Technologies· pp. 1707-1712· 0 citations· 20 references
Abstract
Coordinating heterogeneous aerial and ground vehicles with fundamentally different dynamic time-scales is a longstanding challenge in multi-agent robotics. This paper proposes a distributed reinforcement learning (DRL) framework for cooperative Unmanned Aerial Vehicle (UAV) and Unmanned Ground Vehicle (UGV) formation control that explicitly accounts for multirate dynamics: UAVs execute high-frequency control at 10 Hz while UGVs operate at 2 Hz. Each agent maintains a local actor-critic network trained via a variant of MultiAgent Proximal Policy Optimisation (MAPPO) augmented with a consensus-based communication graph and a multirate synchronisation module that bridges the temporal mismatch between the two vehicle classes. A shaped reward formulation penalises formation deviation, inter-agent collisions and communication dropout simultaneously. Simulation experiments In a physicsaccurate Gazebo/ROS2 environment with two UAV leaders, two UAV followers, and two UGVs demonstrate a task success rate of 96.4%, a formation accuracy of 97.1%, and a collision rate of only 0.8%, outperforming MADDPG, MAPPO (singlerate), centralised DDPG and PID baselines by margins of up to 24.9 percentage points. The framework is further validated in a disaster-response scenario with a dynamic obstacle fields confirming its suitability for real-world heterogeneous multirobot missions.
An efficient way to resolve the curse of dimensionality, improve obstacle avoidance and cooperative formation control of UAVs was found and shows great prospects of practical application in such domains as military operations, search and rescue missions, transport automation and disaster management.
Qadir Talibov· Problems of Information Tech...· 0 citations
Safe cooperative navigation of unmanned aerial vehicle (UAV) formations through three-dimensional environments with dense obstacles, dynamic threats, and unstructured terrain requires jointly addressing goal-directed navigation, formation keeping, hard safety constraints, and formation-topology consistency. We present FALCON-MASAC, a safety-integrated multi-agent reinforcement learning framework that decomposes this task into four complementary layers: (1) a hierarchical leader-follower paradigm that pairs a pre-trained virtual leader with followers learning a distributed cooperative policy; (2) a dual-scale entity-risk attention encoder (DSER-AE) that structures heterogeneous observations into entity-scale and risk-scale semantic tokens and fuses them through intra-scale self-attention and inter-scale full-sequence self-attention over the concatenated tokens; (3) a safety shield built on signed distance functions and high-order control barrier functions (SDF-HOCBF) that maintains the conditional safety certificate while the robust quadratic program remains feasible and the high-order admissibility conditions hold, and switches to a bounded best-effort fallback otherwise; and (4) a bypass-side commitment coordination layer that suppresses trajectory chattering and mitigates crossing conflicts among neighboring UAVs. The safety analysis explicitly quantifies the SDF linearization error and dynamic-obstacle prediction uncertainty and describes the bounded fallback used when certified execution is unavailable. Under the centralized-training-with-decentralized-execution (CTDE) paradigm, simulation experiments show that FALCON-MASAC substantially outperforms representative baselines: it attains a 97.3% success rate (21.7 percentage points above the strongest pure-MARL baseline and 13.0 percentage points above a post-hoc CBF-RL variant), a success-conditional steady-tracking formation error of 1.524 m, and a minimum clearance of 2.732 m. The framework also generalizes well along two out-of-distribution dimensions: obstacle density and dynamic-obstacle speed.
Yiming Shang, Changping Du, Rui Yang et al.· Unmanned Systems· 0 citations
Unmanned aerial vehicles (UAVs) have emerged as a key enabler of next-generation Internet of Things (IoT) ecosystems, offering flexible aerial relaying to extend connectivity across dynamic vehicular ad hoc networks (VANETs) in smart city environments. However, conventional centralized approaches for UAV trajectory planning require continuous global network state aggregation, making them impractical under bandwidth and energy constraints typical of dense urban deployments. In this article, we present TRUAV, a distributed multi-agent reinforcement learning framework based on independent tabular Q-learning for joint UAV trajectory planning and routing enhancement in UAV-aided VANETs. Each UAV is equipped with a local Q-learning agent that operates purely on locally observable information, including vehicle density, packet queue states, and neighbor UAV positions, thereby eliminating the need for global state exchange. A potential-game-inspired reward design encourages spatial diversity and routing-aware UAV positioning among interacting agents while accounting for energy consumption. Numerical simulations over a large urban area with 200 mobile vehicles show that the proposed TRUAV framework achieves network coverage and packet delivery ratios comparable to centralized deep reinforcement learning methods, while also improving relay delay and energy efficiency. Finally, we discuss emerging challenges and future research directions for distributed multi-agent UAV-assisted IoT systems.
Muhammad Umar Farooq Qaisar, Lin Zhang, Zhen Chen et al.· arXiv.org· 0 citations
A cooperative guidance law based on the experience-guided multi-agent proximal policy optimization (E-MAPPO) algorithm is proposed for multiple unmanned aerial vehicles (UAVs) to track dynamic points of interest in civilian applications and results indicate that the proposed method generalizes well to different types of maneuvering targets.
Hao Xiong, Minghu Tan, Xiaoyu Liu et al.· Drones· 0 citations
A distributed real-time trajectory-planning method that integrates a distributed model predictive control framework with an adaptive Gaussian collocation strategy (DA-GCMPC) was developed, which achieves lower computation time and better trajectory quality metrics under the tested simulation settings.
Yang Zhao, Mingying Huo, Naiming Qi et al.· Drones· 0 citations
A multi-agent deep reinforcement learning framework that addresses issues through coordinated exploration, demonstration exploitation, safe curriculum scheduling, and structure-aware generalisation is proposed, demonstrating strong performance in collaboration success rate, navigation robustness, zero-shot cross-scenario generalisation, and dynamic environment adaptability.
Yuhuang Su, Nabil Aouf· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
