Jul 2026· International Conference on Smart Communications and Networking· pp. 1-6· 0 citations· 15 references
Abstract
Unmanned Aerial Vehicle (UAV) swarm networks demand routing protocols that adapt continuously to rapid topology changes, node mobility, and fluctuating link quality. AODV may incur route-discovery overhead after topology changes, while OLSR relies on periodic topology dissemination that may lag behind fast link-quality changes; both can struggle under UAV swarm dynamics. We present RoutePPO, a closedloop adaptive routing framework that couples Proximal Policy Optimization (PPO) with eBPF-based real-time link telemetry and a P4 programmable data plane. RoutePPO-Adapt introduces a Top-K path encoder with fixed-order slot assignment and a 3-step slot-history observation, producing a topology-agnostic 30-dimensional state representation. Training uses a 9-scenario curriculum with anticipatory reward shaping and cosine learningrate decay. Across 15 deterministic routing scenarios, RoutePPOAdapt achieves a mean reward of 0.674–9.2% above the two-path baseline (RoutePPO-Base) - winning 11 of 15 scenarios while reducing latency by 31.6% and packet loss by 37.1%. A kernel-native evaluation (Linux netns + eBPF TC egress) confirms non-zero telemetry counters (0.13-0.27 Mbps), demonstrating end-to-end viability of the eBPF-PPO pipeline.
This study explores the joint optimization of decentralized opportunistic routing and controllable unmanned aerial vehicle (UAV) flight, aiming to enlarge future contacts through discrete UAV headings while enabling per-node replication under contact-limited observations.
This study proposes an intelligent Q-learning-enhanced Evolutionary Game Theory (QEGT) routing mechanism for USNs that leverages game-theoretic incentives and Q-learning to adaptively select strategies.
Anita Murmu, Saurabh Kumar Srivastava, Nuthan Chingeetham et al.· IEEE Open Journal of the Com...· 0 citations
Unmanned aerial vehicles (UAVs) have emerged as a key enabler of next-generation Internet of Things (IoT) ecosystems, offering flexible aerial relaying to extend connectivity across dynamic vehicular ad hoc networks (VANETs) in smart city environments. However, conventional centralized approaches for UAV trajectory planning require continuous global network state aggregation, making them impractical under bandwidth and energy constraints typical of dense urban deployments. In this article, we present TRUAV, a distributed multi-agent reinforcement learning framework based on independent tabular Q-learning for joint UAV trajectory planning and routing enhancement in UAV-aided VANETs. Each UAV is equipped with a local Q-learning agent that operates purely on locally observable information, including vehicle density, packet queue states, and neighbor UAV positions, thereby eliminating the need for global state exchange. A potential-game-inspired reward design encourages spatial diversity and routing-aware UAV positioning among interacting agents while accounting for energy consumption. Numerical simulations over a large urban area with 200 mobile vehicles show that the proposed TRUAV framework achieves network coverage and packet delivery ratios comparable to centralized deep reinforcement learning methods, while also improving relay delay and energy efficiency. Finally, we discuss emerging challenges and future research directions for distributed multi-agent UAV-assisted IoT systems.
Muhammad Umar Farooq Qaisar, Lin Zhang, Zhen Chen et al.· arXiv.org· 0 citations
Timely delivery of delay-sensitive information over dynamic, heterogeneous networks is essential for NextG interactive applications, yet providing strict End-to-End (E2E) peak latency guarantees remains an open challenge. Two obstacles limit the adoption of learning-based network control in this setting: traditional volume-based routing metrics, while highly effective for general traffic management, are not designed to capture traffic urgency; and Deep Reinforcement Learning (DRL) controllers trained from scratch suffer from sample inefficiency, long training times, and early-stage exploration volatility. This paper introduces a deployment-focused network control framework that addresses both obstacles. First, we present Effective Congestion (EC), a deadline-aware metric family that quantifies interface congestion by packet urgency and proactively filters non-viable traffic, coupled with a Uniform Path Grouping (UPG) distribution heuristic promoting robust load-balancing; the resulting policies are embedded into Multi-Agent Deep Reinforcement Learning Effective Congestion ($p^*$) (MADRL EC ($p^*$)), a hybrid architecture combining a distributed scheduler with a centralized RL-based router. Second, we introduce a unified training objective that generalizes existing policy-learning paradigms---behavioral cloning, offline Reinforcement Learning (RL), online RL, and offline-to-online schemes---as special cases, combining a live-reward term, a pre-collected-reward term, and a policy-imitation term. From this objective, we derive the Model-Guided Annealed Reinforcement Learning (MGA-RL) protocol, instantiated on a Deep Deterministic Policy Gradient (DDPG) backbone: a deployment-oriented, demonstration-driven training approach that generalizes conventional Offline-to-Online (O2O) schemes, in which trajectories from a lightweight [...]
Vincenzo Norman Vitale, Mohammad Solki, A. Tulino et al.· 0 citations
High-performance networking is essential for Unmanned Aerial Vehicle (UAV) swarms to accomplish complex, coordinated missions. A central challenge in UAV swarm networking is managing concurrent multi-hop transmissions, where traditional protocols often struggle due to routing path conflicts and co-channel interference. To address this, we propose a novel multi-agent reinforcement learning (MARL)-based cross-layer transmission framework that maximizes system throughput by jointly optimizing network-layer routing, link-layer resource allocation, and UAV trajectories. We decouple this complex joint optimization problem and solve it with a routing-prioritized iterative scheme. For the routing sub-problem, an MARL approach is designed for agents to collaboratively plan concurrent routing paths. The non-convex resource allocation and trajectory sub-problems are handled using successive convex approximation (SCA). Experimental results demonstrate that our proposed framework significantly outperforms existing benchmarks in system throughput, end-to-end delay, and packet delivery ratio.
Yang Shen, Bing Li, Rong-Qing Zhang· IEEE Transactions on Wireles...· 0 citations
Future sixth-generation (6G)-oriented networks require programmable control that can adapt routing to latency and congestion without unsafe online exploration. This study evaluates offline multi-agent deep deterministic policy gradient (MADDPG) with behavior-adjusted training rewards for latency-aware path control in software-defined networking (SDN). Each traffic pair is modeled as an agent selecting one of three retained candidate paths, while centralized critics learn coordinated decisions from topology-specific Ryu–Mininet transition datasets. Nine policies are compared using ten paired seeds on fat-tree, mesh-grid, and WAN-corridors topologies under a deployed utilization–latency weighting of 0.60/0.40, together with flow-completion, latency, congestion, architectural-comparison, sensitivity, robustness, statistical, and controller-overhead analyses. The utilization-aware path heuristic achieves the strongest overall reward ranking. MADDPG is the strongest learned policy on fat-tree, is not significantly outperformed by any evaluated policy on mesh-grid, and remains statistically tied with completion-matched policies on WAN-corridors. Behavior adjustment is topology-dependent rather than uniformly beneficial. The exported policy requires approximately 52μs per joint decision, whereas complete control-loop timing is dominated by network-statistics polling. These results support offline multi-agent SDN control as a competitive, low-overhead option when interpreted jointly with topology structure, flow completion, and strong heuristic baselines.
A. Kyzyrkanov, Y. Nurakhov, Zhenis Otarbay et al.· Technologies· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
