This study explores the joint optimization of decentralized opportunistic routing and controllable unmanned aerial vehicle (UAV) flight, aiming to enlarge future contacts through discrete UAV headings while enabling per-node replication under contact-limited observations.
Abstract
The growing deployment of delay-tolerant networks (DTNs) has made store-carry-forward (SCF) communication indispensable under sparse connectivity. However, intermittent contacts, finite buffers, and limited message time-to-live (TTL) often give rise to sparse delivery and congestion, leading to substantial end-to-end performance degradation. To address this challenge, this study explores the joint optimization of decentralized opportunistic routing and controllable unmanned aerial vehicle (UAV) flight, aiming to enlarge future contacts through discrete UAV headings while enabling per-node replication under contact-limited observations. Building upon this architecture, we study cooperative factored routing--UAV control under centralized training and decentralized execution (CTDE) and propose JUROR (Joint UAV flight and Opportunistic Routing, based on the proximal policy optimization (PPO) framework. In our design, we first cast the problem as a factored partially observable Markov decision process with sequential motion--routing coupling and a per-step team reward; subsequently, decentralized actors act on local observations while a training-time critic uses global statistics, and an optional multi-horizon hotspot predictor provides auxiliary supervision. Simulation results over four traffic modes demonstrate effective gains over PRoPHET and MaxProp, while retaining contact-limited decentralized execution.
The growing demand for multimedia services in Internet of Things (IoT) networks has significantly increased the traffic load on backhaul links, making Mobile Edge Caching (MEC) a key technology for reducing content delivery latency. Unmanned Aerial Vehicles (UAVs) can serve as mobile aerial caching nodes that complement fixed ground infrastructure, but their small cache size and limited battery life restrict how long and how effectively they can operate. In addition, current approaches often optimize caching decisions, user association, and flight trajectories separately, without considering their interactions under tight energy constraints. In this paper, we formulate a joint optimization problem that aims to minimize the average content retrieval delay in an energy-constrained multi-UAV cooperative caching system. We then propose a deep reinforcement learning (DRL) framework based on the Multi-Agent Twin Delayed Deep Deterministic Policy Gradient (MATD3) algorithm, in which each UAV is trained as an independent agent under a centralized-training and decentralized-execution scheme. Simulation results show that our method outperforms several heuristic and non-cooperative reinforcement learning baselines in terms of cache hit rate and energy efficiency. Specifically, the proposed method reduces the system's average content retrieval delay with a maximum reduction of 9.1% and effectively guarantees an average cache hit rate of 62.45%, maintaining a sustained remaining energy margin over baseline methodologies.
Tao Zhang, Tao Xu, Zekai Liu et al.· Journal of Circuits, Systems...· 0 citations
Unmanned aerial vehicles (UAVs) have emerged as a key enabler of next-generation Internet of Things (IoT) ecosystems, offering flexible aerial relaying to extend connectivity across dynamic vehicular ad hoc networks (VANETs) in smart city environments. However, conventional centralized approaches for UAV trajectory planning require continuous global network state aggregation, making them impractical under bandwidth and energy constraints typical of dense urban deployments. In this article, we present TRUAV, a distributed multi-agent reinforcement learning framework based on independent tabular Q-learning for joint UAV trajectory planning and routing enhancement in UAV-aided VANETs. Each UAV is equipped with a local Q-learning agent that operates purely on locally observable information, including vehicle density, packet queue states, and neighbor UAV positions, thereby eliminating the need for global state exchange. A potential-game-inspired reward design encourages spatial diversity and routing-aware UAV positioning among interacting agents while accounting for energy consumption. Numerical simulations over a large urban area with 200 mobile vehicles show that the proposed TRUAV framework achieves network coverage and packet delivery ratios comparable to centralized deep reinforcement learning methods, while also improving relay delay and energy efficiency. Finally, we discuss emerging challenges and future research directions for distributed multi-agent UAV-assisted IoT systems.
Muhammad Umar Farooq Qaisar, Lin Zhang, Zhen Chen et al.· arXiv.org· 0 citations
High-performance networking is essential for Unmanned Aerial Vehicle (UAV) swarms to accomplish complex, coordinated missions. A central challenge in UAV swarm networking is managing concurrent multi-hop transmissions, where traditional protocols often struggle due to routing path conflicts and co-channel interference. To address this, we propose a novel multi-agent reinforcement learning (MARL)-based cross-layer transmission framework that maximizes system throughput by jointly optimizing network-layer routing, link-layer resource allocation, and UAV trajectories. We decouple this complex joint optimization problem and solve it with a routing-prioritized iterative scheme. For the routing sub-problem, an MARL approach is designed for agents to collaboratively plan concurrent routing paths. The non-convex resource allocation and trajectory sub-problems are handled using successive convex approximation (SCA). Experimental results demonstrate that our proposed framework significantly outperforms existing benchmarks in system throughput, end-to-end delay, and packet delivery ratio.
Yang Shen, Bing Li, Rong-Qing Zhang· IEEE Transactions on Wireles...· 0 citations
This paper proposes AERIS, an offline policy improvement framework for multi-UAV ISAC that learns from fixed flight logs under centralized training and decentralized execution and designs STAR-CRDT, an offline multi-agent RL algorithm that performs support-aware local action rectification and distills only trusted improvements into the decentralized actor.
Ziyuan Wang, Yi-Fan Sui, Wei Wei et al.· 0 citations
6G mobile edge networks are emerging as a key infrastructure for ubiquitous large language model (LLM) inference services. However, conventional edge routing to nearby or well-connected servers falls short for efficient edge LLM inference, as it may miss the user’s KV cache and trigger costly prefill recomputation. To address this challenge, this paper studies an edge inference system assisted by an embodied UAV agent swarm, where UAVs actively sense user mobility and neighboring UAV states to make local decisions on trajectory control, user association, and inference-request routing. The goal is to improve KV-cache reuse while maintaining reliable wireless connectivity, thereby maximizing the system effective token throughput under energy and QoS constraints. We then formulate the joint optimization as a mixed-integer non-linear program and further cast the sequential UAV decision-making process as a decentralized partially observable Markov decision process. To obtain scalable decentralized policies under partial observations, we propose Q-MAA2C, a quantum-enhanced multi-agent advantage actor-critic algorithm for embodied UAV swarm control and inference routing. Q-MAA2C uses quantum actors for local action selection and an entangled split critic for swarm-level value estimation, enabling coordinated policies from partial observations with reduced raw observation exchange. Simulation results indicate that Q-MAA2C yields comparable reinforcement learning rewards to the fully classical baseline while reducing the number of convergence episodes by about 43%. Additionally, the proposed method enhances the system effective token throughput by up to about 134% over other competing methods.
Xiangdong Zheng, Long Luo, Hongfang Yu et al.· IEEE Transactions on Cogniti...· 1 citation
This paper proposes a heterogeneous multi-agent proximal policy optimization (MAPPO)-based framework where both user devices and UAVs act as heterogeneous agents and utilizes a centralized training and decentralized execution (CTDE) paradigm to enable collaborative strategies between computing requesters and providers.
Ming Cheng, Canlin Zhu, Jiang-Hang Tang et al.· Journal of King Saud Univers...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
