Findings indicate that directional pheromone-gradient observations provide an effective and communication-efficient mechanism for decentralized swarm coordination, improving search effectiveness and operational robustness in post-disaster SAR scenarios.
Abstract
Search-and-rescue (SAR) operations in disaster environments require drone swarms to coordinate efficiently despite incomplete information and potential communication failures. Existing stigmergy-based approaches provide low-bandwidth coordination but rely on fixed rules, whereas multi-agent reinforcement learning (MARL) can learn adaptive behaviors but often struggles with coordination under partial observability. To address these limitations, this paper proposes a Hybrid stigmergy–MARL framework that introduces directional pheromone-gradient observations, enabling each drone to infer the direction of likely victims and unexplored regions using locally available information. The proposed framework combines reinforcement learning with four virtual pheromone layers representing coverage history, victim likelihood, environmental risk, and communication quality. Victim detection is modeled through an abstract short-range thermal/visual sensing mechanism, while environmental information is shared through pheromone-based environmental memory to reduce dependence on direct communication. The simulated environment consists of a 40 × 40 grid, where each grid cell represents a discrete two-dimensional location. Victims occupy a single grid cell, and obstacles are modeled as static two-dimensional impassable cells. Experimental results show that the proposed approach achieved 98.9% area coverage and 93.3% victim detection, compared with 81.8% coverage and 71.7% victim detection for the RL-only baseline. Ablation experiments confirmed that directional gradient observations are the primary contributor to these improvements, while communication-loss experiments demonstrated robust performance even under complete communication outage. These findings indicate that directional pheromone-gradient observations provide an effective and communication-efficient mechanism for decentralized swarm coordination, improving search effectiveness and operational robustness in post-disaster SAR scenarios.
The proposed framework demonstrates robust scalability and real-time coordination capability for dynamic environments, while providing a reliable decision-making paradigm for intelligent multi-agent systems operating in communication-intensive and electromagnetically complex application scenarios.
X.-H. Fang, K. Chen, Cheng-Hao Ren et al.· Advanced Electromagnetics· 0 citations
A multi-agent deep reinforcement learning framework that addresses issues through coordinated exploration, demonstration exploitation, safe curriculum scheduling, and structure-aware generalisation is proposed, demonstrating strong performance in collaboration success rate, navigation robustness, zero-shot cross-scenario generalisation, and dynamic environment adaptability.
The increasing frequency and intensity of wildfires has created an urgent demand for scalable and autonomous wildfire response systems. While recent advances in multi-agent reinforcement learning (MARL) have demonstrated promise for collaborative uncrewed aerial vehicle (UAV)-based wildfire suppression, most existing approaches rely on simplified fire propagation dynamics and highly centralised learning architectures that are difficult to deploy in realistic operational settings. This paper presents a decentralised MARL framework for wildfire suppression that combines stochastic wildfire propagation, wind-driven spread dynamics, and communication-aware multi-agent coordination. The proposed framework extends an existing probabilistic wildfire environment through the incorporation of wind speed and directional effects, producing highly asymmetric and stochastic wildfire behaviour that more closely resembles real wildfire propagation. A decentralised Deep Q-Network (DQN) architecture is then introduced in which UAV agents learn independently through individual replay buffers. To mitigate the sparse-learning challenges introduced by decentralisation, selective experience sharing based on the SUPER algorithm is incorporated, enabling agents to exchange only high-value experiences under realistic communication constraints. Experimental results demonstrate that selective communication significantly improves containment performance and learning efficiency while preserving decentralised execution. The work highlights both the feasibility and challenges of realistic UAV swarm coordination for wildfire suppression, particularly the trade-offs between communication bandwidth, environmental stochasticity, and collaborative performance.
S. Cartwright, Maxime Collignon, Adolfo Perrusquía et al.· Drones· 0 citations
Multi-UAV Cooperative Target Search (MCTS) is a critical task in low-altitude sensing applications, requiring agents to efficiently explore unknown environments under complex constraints. However, traditional search methods are mostly unscalable and perform poorly in dynamic multi-UAV environments. As a promising alternative, Reinforcement Learning (RL) has emerged to overcome these limitations by enabling agents to learn adaptive policies directly from environmental interactions. A key limitation is that current RL methods lack efficient exploration, which is a critical bottleneck preventing UAVs from finding more targets. To address this limitation, we propose a novel method named AEQMIX, which integrates trajectory entropy maximization into QMIX, an advanced Multi-Agent Reinforcement Learning (MARL) method, to encourage efficient exploration. We formulate the MCTS problem as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP) and design a multi-objective reward function. To mitigate the intractability of density estimation in high-dimensional spaces, we employ a nonparametric particle-based entropy estimator to quantify the spatial diversity of UAV trajectories. This entropy estimate is utilized as an intrinsic reward, incentivizing agents to maximize the distance between their trajectories and those of their neighbors. Extensive simulations demonstrate that AEQMIX significantly outperforms baseline reinforcement learning and traditional optimization methods in terms of search rate, coverage efficiency, and collision avoidance. Compared with DNQMIX, AEQMIX improves the search rate and coverage rate by 9.52% and 11.54%, respectively, while reducing the average collision count by 70.59% in the (40 × 40) environment.
The field of swarm robotics has become more and more popular as a decentralised solution to problems of coordination among multiple independent agents. The recent developments in multi-agent reinforcement learning (MARL) have made it possible for agents to learn cooperative behaviours when operating in a dynamic environment and even outperform the traditional rule-based or heuristic coordination strategies. However, coordination in practical MARL-based swarms is still difficult, as many approaches are not scalable, have high communication cost, unstable coordination with high swarm density, and lack integration of efficiency, robustness, and adaptability. This paper introduces a multi-agent reinforcement learning framework for scalable and communication-efficient swarm coordination called RoboSwarmCoordAI, which surpasses the limitations of the above approaches. The proposed framework uses three major components: a state-encoding module that is aware of the coordination requirements, an adaptive neighbourhood-filtering module to avoid redundant inter-agent communication and a hybrid reward function that weights local robot goals with respect to the global swarm performance. RoboSwarmCoordAI uses a centralised training and decentralised execution approach where agents can leverage global information for training, but local information for execution. The framework was tested in simulation in cooperative exploration, distributed target search and task allocation scenarios. When evaluated within the range of simulations tested, RoboSwarmCoordAI outperformed baseline methods with a task success rate of 95.8%, 2.7 collisions per episode, and an efficiency score of 91.6. The analyses of scalability and communication efficiency also demonstrate the stable coordination performance up to 50 robots, at a lower communications cost. As a result, it was found that RoboSwarmCoordAI is a promising simulation-validated framework for adaptive swarm coordination, and future work will further validate RoboSwarmCoordAI on larger swarms and physical robotic platforms.
S. Vinta, Rakesh Phanindra Akula, Surya Samantha Beri et al.· Discover Computing· 0 citations
IB-CEMARL is proposed, an information-bottleneck-guided, communication-efficient multi-agent reinforcement learning framework for UAV swarms that achieves superior cooperative performance, reduced message redundancy, and stronger robustness compared with representative communication-aware MARL baselines.
