2026· IEEE Transactions on Cognitive Communications and Networking· Vol 12, pp. 10607-10622· 0 citations· 43 references
Computer Science
Abstract
Rapid advances in drone technology, combined with the growing congestion of terrestrial transport networks, are driving the emergence of the low-altitude economy. Uncrewed Aerial Vehicles (UAVs) are increasingly deployed for low-altitude economy applications such as urban logistics and transportation, yet their expansion is constrained by the scarcity of spectrum resources. Although Multi-Agent Reinforcement Learning (MARL) offers a promising decentralized approach to improve spectral efficiency of UAVs, existing MARL methods suffer from high training costs, often requiring extensive environmental interactions. To overcome these limitations, we propose a novel Multi-Agent Model-Based reinforcement learning algorithm for decentralized spectrum sharing among UAVs in the low-altitude economy, which we denote as MAMBA-UAV. Adopting a Centralized Training with Decentralized Execution (CTDE) paradigm, MAMBA-UAV equips each UAV with a learned world model that captures compact environmental representations and predicts system dynamics. These world models are then utilized during MARL training to simulate interactions, thereby reducing the reliance on repeated real-environment rollouts. Through comprehensive simulations, we demonstrate that MAMBA-UAV substantially reduces the number of environmental interactions required for UAVs to achieve competitive spectrum-sharing performance, lowering training costs while maintaining high performance.
Multi-UAV Cooperative Target Search (MCTS) is a critical task in low-altitude sensing applications, requiring agents to efficiently explore unknown environments under complex constraints. However, traditional search methods are mostly unscalable and perform poorly in dynamic multi-UAV environments. As a promising alternative, Reinforcement Learning (RL) has emerged to overcome these limitations by enabling agents to learn adaptive policies directly from environmental interactions. A key limitation is that current RL methods lack efficient exploration, which is a critical bottleneck preventing UAVs from finding more targets. To address this limitation, we propose a novel method named AEQMIX, which integrates trajectory entropy maximization into QMIX, an advanced Multi-Agent Reinforcement Learning (MARL) method, to encourage efficient exploration. We formulate the MCTS problem as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP) and design a multi-objective reward function. To mitigate the intractability of density estimation in high-dimensional spaces, we employ a nonparametric particle-based entropy estimator to quantify the spatial diversity of UAV trajectories. This entropy estimate is utilized as an intrinsic reward, incentivizing agents to maximize the distance between their trajectories and those of their neighbors. Extensive simulations demonstrate that AEQMIX significantly outperforms baseline reinforcement learning and traditional optimization methods in terms of search rate, coverage efficiency, and collision avoidance. Compared with DNQMIX, AEQMIX improves the search rate and coverage rate by 9.52% and 11.54%, respectively, while reducing the average collision count by 70.59% in the (40 × 40) environment.
Unmanned aerial vehicles (UAVs) have emerged as a key enabler of next-generation Internet of Things (IoT) ecosystems, offering flexible aerial relaying to extend connectivity across dynamic vehicular ad hoc networks (VANETs) in smart city environments. However, conventional centralized approaches for UAV trajectory planning require continuous global network state aggregation, making them impractical under bandwidth and energy constraints typical of dense urban deployments. In this article, we present TRUAV, a distributed multi-agent reinforcement learning framework based on independent tabular Q-learning for joint UAV trajectory planning and routing enhancement in UAV-aided VANETs. Each UAV is equipped with a local Q-learning agent that operates purely on locally observable information, including vehicle density, packet queue states, and neighbor UAV positions, thereby eliminating the need for global state exchange. A potential-game-inspired reward design encourages spatial diversity and routing-aware UAV positioning among interacting agents while accounting for energy consumption. Numerical simulations over a large urban area with 200 mobile vehicles show that the proposed TRUAV framework achieves network coverage and packet delivery ratios comparable to centralized deep reinforcement learning methods, while also improving relay delay and energy efficiency. Finally, we discuss emerging challenges and future research directions for distributed multi-agent UAV-assisted IoT systems.
Muhammad Umar Farooq Qaisar, Lin Zhang, Zhen Chen et al.· arXiv.org· 0 citations
Unmanned Aerial Vehicles (UAVs) have gained widespread attention in diverse applications like military, medical, aerial surveillance and many more. Presently, the problem of limited bandwidth and geographic factors has raised the need for effective and timely data transfer. Training UAVs with reinforcement learning-based algorithms facilitates autonomous decision-making capabilities. In this paper, we proposed an intelligent system for the optimal UAV selection process by evaluating the continuous performance of each UAV. The analyzing factors are based on the real-world factors affecting the quality of signals, such as noise interference, relative motion between source and wave, and transmission power. Based on the systematic conditions observed, the system provides efficient rewards. To promote the selection of the optimal UAV and enhance the learning process, the state information of the UAV is fed into a deep neural network (DQN), which predicts the 'Q-values'. Our system implements a deep Q-learning algorithm, which enhances the agent's performance by systematically learning from its experience. The model operates accurately by selecting the most reliable UAV, thus, enhancing the throughput by optimal power allocation. It outperforms other conventional models in terms of timely data delivery and energy utilization. The system adapts various complex patterns by analyzing the historical and present scenarios. Empowered by this intelligent system, time-critical decision-making can be achieved with minimal energy consumption.
Divyanshu Bhardwaj, Angel Kanjiya, N. Jadav et al.· 2026 IEEE International Work...· 0 citations
The increasing frequency and intensity of wildfires has created an urgent demand for scalable and autonomous wildfire response systems. While recent advances in multi-agent reinforcement learning (MARL) have demonstrated promise for collaborative uncrewed aerial vehicle (UAV)-based wildfire suppression, most existing approaches rely on simplified fire propagation dynamics and highly centralised learning architectures that are difficult to deploy in realistic operational settings. This paper presents a decentralised MARL framework for wildfire suppression that combines stochastic wildfire propagation, wind-driven spread dynamics, and communication-aware multi-agent coordination. The proposed framework extends an existing probabilistic wildfire environment through the incorporation of wind speed and directional effects, producing highly asymmetric and stochastic wildfire behaviour that more closely resembles real wildfire propagation. A decentralised Deep Q-Network (DQN) architecture is then introduced in which UAV agents learn independently through individual replay buffers. To mitigate the sparse-learning challenges introduced by decentralisation, selective experience sharing based on the SUPER algorithm is incorporated, enabling agents to exchange only high-value experiences under realistic communication constraints. Experimental results demonstrate that selective communication significantly improves containment performance and learning efficiency while preserving decentralised execution. The work highlights both the feasibility and challenges of realistic UAV swarm coordination for wildfire suppression, particularly the trade-offs between communication bandwidth, environmental stochasticity, and collaborative performance.
S. Cartwright, Maxime Collignon, Adolfo Perrusquía et al.· Drones· 0 citations
This work investigates a reinforcement learning-based control framework for the autonomous movement and coordination of multiple Unmanned Aerial Vehicles (UAVs) in a wireless communication environment. The considered system includes UAVs performing sensing and relaying tasks, where mobility decisions directly affect the overall network performance. The main objective is to improve the communication quality of ground users by maximizing aggregate network throughput. To achieve this objective, a Double Deep Q-Network (DDQN) architecture is employed, where each UAV is assigned an individual learning agent. The agents learn role-specific movement policies while coordinating through interactions with the shared environment. Learning performance is further improved by using adaptive scaling and a custom reward function designed to capture variations in network utility. Simulation results show that the proposed approach outperforms baseline movement strategies in terms of utility. In addition, different task configurations, agent behaviors, and hyperparameter selections are examined to improve convergence speed and training stability. Overall, the results indicate that reinforcement learning is a promising method for cooperative UAV positioning in dynamic and interference-sensitive wireless communication scenarios.
Berke Kilinç, M. Ö. Efe· International Conference on...· 0 citations
Ensuring QoS provisioning in low-altitude wireless networks requires UAV positioning and navigation strategies that adapt to dynamic environments and generalizes across heterogeneous network scenarios. This paper proposes a digital twin (DT)-assisted meta reinforcement learning framework for multi-agent UAV trajectory planning. A high-fidelity network DT serves as a supervisory layer to generate key performance indicators (KPIs) and fine-grained channel knowledge, which guides both domain-specific learning and cross-domain validation. Building on the twin-informed UAV landmarks, we then develop a weakness-aware meta learning scheme: in the inner loop, agents are trained cooperatively toward the self-discovered landmarks under dynamic conditions; in the outer loop, navigation policies are evaluated via the DT to identify bottlenecks and generate targeted hard scenarios, enabling robust adaptation across diverse scenarios. Extensive simulations show that our framework achieves up to 4× higher service coverage compared to baselines, while the target-aware outer-loop adaptation further improves cross-scene performance and model generalization.
Jiayuan Huang, E. Tucker, Ruozhou Yu et al.· International Conference on...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
