This paper proposes AERIS, an offline policy improvement framework for multi-UAV ISAC that learns from fixed flight logs under centralized training and decentralized execution and designs STAR-CRDT, an offline multi-agent RL algorithm that performs support-aware local action rectification and distills only trusted improvements into the decentralized actor.
Abstract
Unmanned aerial vehicle (UAV)-enabled integrated sensing and communication (ISAC) is a promising 6G paradigm, but dynamic multi-UAV ISAC control must jointly balance communication quality, sensing reliability, and flight safety under stochastic mobility. Existing optimization methods often require repeated global non-convex solving, while online reinforcement learning (RL) depends on risky trial-and-error flights that may cause sensing loss or collision-risk events. This paper proposes AERIS, an offline policy improvement framework for multi-UAV ISAC. AERIS learns from fixed flight logs under centralized training and decentralized execution, so each UAV acts from local histories while training uses logged global information to assess team-level effects. We further design STAR-CRDT, an offline multi-agent RL algorithm that performs support-aware local action rectification and distills only trusted improvements into the decentralized actor. We prove an offline-support policy improvement guarantee. Experiments show that STAR-CRDT improves the main ISAC objective return by 29.3% over the strongest baseline. It further improves communication sum rate, sensing pass rate, and sensing margin by 3.4%, 4.8%, and 69.1%, while reducing collision-risk events by 54.2%. On unseen real-road maps built from OpenStreetMap data, STAR-CRDT still obtains the best return.
Unmanned Aerial Vehicles (UAVs) are pivotal for facilitating data collection in emergency scenarios. Despite the potential of Multi-Agent Deep Reinforcement Learning (MADRL) in coordinating such systems, existing researches struggle to resolve the high-dimensional coupling of data collection, trajectory planning, and energy scheduling under strict collision avoidance and Return-To-Base (RTB) constraints. This paper proposes a energy-aware cooperative MADRL framework designed to maximize data collection utility under energy constraints. Specifically, we employ a Multi-Agent Deep Deterministic Policy Gradient (MADDPG) approach featuring a Centralized Training with Decentralized Execution (CTDE) design and a multi-objective reward mechanism to balance conflicting optimization goals. Extensive simulations validate the advantages of the proposed framework over leading baselines. Notably, the algorithm exhibits significant quantitative advantages in complex high-load scenarios. These outcomes prove that our method achieves higher task completion rates while strictly adhering to RTB and safety protocols.
Jing Mei, Jing-Lei Xu, Zhao Tong et al.· IEEE Transactions on Network...· 0 citations
Unmanned aerial vehicles (UAVs) have emerged as a key enabler of next-generation Internet of Things (IoT) ecosystems, offering flexible aerial relaying to extend connectivity across dynamic vehicular ad hoc networks (VANETs) in smart city environments. However, conventional centralized approaches for UAV trajectory planning require continuous global network state aggregation, making them impractical under bandwidth and energy constraints typical of dense urban deployments. In this article, we present TRUAV, a distributed multi-agent reinforcement learning framework based on independent tabular Q-learning for joint UAV trajectory planning and routing enhancement in UAV-aided VANETs. Each UAV is equipped with a local Q-learning agent that operates purely on locally observable information, including vehicle density, packet queue states, and neighbor UAV positions, thereby eliminating the need for global state exchange. A potential-game-inspired reward design encourages spatial diversity and routing-aware UAV positioning among interacting agents while accounting for energy consumption. Numerical simulations over a large urban area with 200 mobile vehicles show that the proposed TRUAV framework achieves network coverage and packet delivery ratios comparable to centralized deep reinforcement learning methods, while also improving relay delay and energy efficiency. Finally, we discuss emerging challenges and future research directions for distributed multi-agent UAV-assisted IoT systems.
Muhammad Umar Farooq Qaisar, Lin Zhang, Zhen Chen et al.· arXiv.org· 0 citations
Integrated sensing and communication (ISAC) technology, when deployed on unmanned aerial vehicles (UAVs), enables aerial base stations to simultaneously provide wireless connectivity to ground users and perform environmental sensing through echo signal analysis. However, the broadcast nature of wireless transmission, combined with the line-of-sight (LoS) propagation characteristics of UAVs, increases the risk of passive eavesdropping on transmitted signals during ISAC missions. This paper investigates the joint trajectory design and power allocation (JTDPA) problem for UAV-enabled ISAC systems in environments with multiple mobile ground users and potential eavesdroppers. The proposed approach formulates the optimization problem as a constrained Markov decision process (CMDP), aiming to balance communication rate, secrecy rate, and energy consumption. To address the limitations of existing secure trajectory designs, such as unnecessary energy expenditure and overly conservative avoidance actions, we propose a two-stage (TS) strategy that incorporates the safe twin delayed deep deterministic policy gradient (Safe-TD3) algorithm, referred to as TS-SafeTD3. In the first stage (sensing stage), the UAV navigates toward a user-centric location without communication to enhance initial coverage efficiency, while satisfying the minimum-distance safety constraints with respect to potential eavesdroppers.In the second stage (ISAC stage), Safe-TD3 is employed to jointly optimize both trajectory and power allocation under the same safety constraints to maximize the weighted secrecy rate. Simulation results indicate that the proposed algorithm improves the weighted secrecy rate and energy efficiency under various operational conditions, while maintaining a low violation probability of the safety constraints.
Yu-Jia Chen, Hai-Yan Huang, Ting-Wei Chen et al.· IEEE Transactions on Network...· 0 citations
Unmanned aerial vehicle (UAV)-assisted mobile edge computing (MEC) systems provide flexible computing services for resource-constrained devices, but malicious jamming attacks introduce dynamic channel conditions and resource competition, making joint trajectory and resource optimization challenging. This paper investigates this problem in multi-UAV MEC systems under jamming, aiming to minimize delay and energy consumption while ensuring anti-jamming robustness. The problem is formulated as a decentralized partially observable Markov decision process (Dec-POMDP). However, traditional multi-agent reinforcement learning (MARL) approaches struggle with high exploration costs and low sampling efficiency in high-dimensional hybrid action spaces. To overcome these limitations, we propose an LLM-guided MARL framework instantiated with the multi-agent deep deterministic policy gradient (MADDPG), which leverages LLM-generated semantic trajectory prompts to dynamically constrain exploration within the continuous action space, effectively compressing the policy search space and accelerating convergence. Simulation results demonstrate that the proposed method achieves $3.4\times $ to $5\times $ faster convergence over hierarchical MADDPG, MADDPG, and independent soft actor-critic (ISAC) baselines, significantly reducing training costs while maintaining superior performance and anti-jamming robustness.
Yeguang Qin, Jie Tang, Fengxiao Tang et al.· IEEE Transactions on Communi...· 0 citations
A service-driven regional partitioning scheme is proposed to support traffic-aware UAV communication, and an adaptive handshaking mechanism is introduced to improve cooperative sensing accuracy by mitigating residual inter-region phase errors with controlled synchronization overhead.
The three-layer temporal decoupling architecture provides a practical template for safely integrating LLM-assisted planning with real-time multi-agent control in contested environments.
Xingda Li, Jian-Qiang Zhang, Yiping Liu et al.· Drones· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
