Jul 2026· Fall Joint Computer Conference· pp. 313-320· 0 citations· 30 references
Abstract
Cooperative autonomous underwater vehicles (AUVs) provide an effective platform for marine environmental monitoring and offshore energy exploration. However, federated learning in constrained underwater networks is challenged by the deep coupling between scarce high-value observations and heterogeneous non-IID sensing data. Existing methods usually ignore the coupling between physical energy consumption and model learning, or aggregate sparse heterogeneous updates uniformly, which may cause inefficient training and pseudo-convergence. To address these challenges, this paper proposes an Energy-Aware Federated Multi-Agent Learning framework, named Energy-Aware FedMARL. The proposed framework formulates cooperative sensing as a constrained Markov decision process and adopts a MATD3-based multi-agent strategy to guide AUVs toward spatially complementary high-value observations under energy constraints. In addition, a quality-aware aggregation mechanism is developed to emphasize sparse local updates containing more high-value samples, with extensions to validation-gain and contribution-aware weighting. Simulation results show that Energy-Aware FedMARL reduces global validation loss by about 30% compared with MAPPO and 68% compared with MADDPG, while improving hotspot discovery and effective federated participation.
Achieving scalable swarm autonomy in Global Navigation Satellite System (GNSS)-denied and communication-constrained environments remains an open challenge at the intersection of robotics, distributed optimization, and reinforcement learning. Existing unmanned aerial vehicle (UAV) autonomy frameworks typically decouple navigation, perception, and distributed learning, while assuming centralized coordination or reliable global positioning. This paper introduces a unified federated deep reinforcement learning architecture that enables GNSS-independent multi-UAV autonomy through the principled integration of multi-modal perception, decentralized policy optimization, energy-aware control, and edge-compliant inference. The proposed framework formulates joint navigation and dynamic target tracking as a partially observable Markov decision process optimized via Proximal Policy Optimization (PPO) over structured motion primitives. A communication-efficient federated learning mechanism enables distributed policy convergence under non-independent and identically distributed (non-IID) agent experiences without sharing raw data, establishing a scalable alternative to centralized training. To address sim-to-real discrepancies, the architecture incorporates domain randomization, structured sensor noise modeling, and curriculum-based training to promote robust zero-shot deployment. Multi-agent simulation experiments evaluate the swarm-level and federated-learning behavior of the proposed framework, while single-UAV field deployment evidence using a DJI Matrice 100 platform supports the feasibility of the onboard sensing, perception, and edge-inference pipeline under realistic outdoor conditions. The evaluation demonstrates stable decentralized convergence, improved energy efficiency relative to centralized baselines, robust target-tracking performance under GNSS-denied conditions, and real-time edge-compliant inference. The results establish that federated reinforcement learning can serve as a viable systems-level foundation for resilient, energy-aware, and scalable aerial swarm intelligence, advancing the state of the art in distributed autonomous robotics.
Nikolaos D. Almalis, G. Tsihrintzis, George Baris et al.· Electronics· 0 citations
The proposed framework is validated by conducting simulation-based experiments on the benchmark datasets and synthetic autonomous workloads, where the novelty lies in the design of the system-level federated learning architecture, instead of the datasets themselves.
Jyotsnarani Tripathy, D. Rajalakshmi, A. N. Ramya Shree et al.· SN Computer Science· 0 citations
Unmanned Aerial Vehicles (UAVs) are pivotal for facilitating data collection in emergency scenarios. Despite the potential of Multi-Agent Deep Reinforcement Learning (MADRL) in coordinating such systems, existing researches struggle to resolve the high-dimensional coupling of data collection, trajectory planning, and energy scheduling under strict collision avoidance and Return-To-Base (RTB) constraints. This paper proposes a energy-aware cooperative MADRL framework designed to maximize data collection utility under energy constraints. Specifically, we employ a Multi-Agent Deep Deterministic Policy Gradient (MADDPG) approach featuring a Centralized Training with Decentralized Execution (CTDE) design and a multi-objective reward mechanism to balance conflicting optimization goals. Extensive simulations validate the advantages of the proposed framework over leading baselines. Notably, the algorithm exhibits significant quantitative advantages in complex high-load scenarios. These outcomes prove that our method achieves higher task completion rates while strictly adhering to RTB and safety protocols.
Jing Mei, Jing-Lei Xu, Zhao Tong et al.· IEEE Transactions on Network...· 0 citations
Results show that topology‐aware cooperative learning can provide a scalable and practical solution for reliable UAV communication in future intelligent aerial and 6G‐enabled networks, particularly in dense deployments.
V. Nam, A. Chehri, Weiwei Jiang et al.· Expert Syst. J. Knowl. Eng.· 0 citations
It is demonstrated that lightweight aggregation strategies can substantially improve empirical safety in federated reinforcement learning while preserving standard communication protocols.
Multi-UAV Cooperative Target Search (MCTS) is a critical task in low-altitude sensing applications, requiring agents to efficiently explore unknown environments under complex constraints. However, traditional search methods are mostly unscalable and perform poorly in dynamic multi-UAV environments. As a promising alternative, Reinforcement Learning (RL) has emerged to overcome these limitations by enabling agents to learn adaptive policies directly from environmental interactions. A key limitation is that current RL methods lack efficient exploration, which is a critical bottleneck preventing UAVs from finding more targets. To address this limitation, we propose a novel method named AEQMIX, which integrates trajectory entropy maximization into QMIX, an advanced Multi-Agent Reinforcement Learning (MARL) method, to encourage efficient exploration. We formulate the MCTS problem as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP) and design a multi-objective reward function. To mitigate the intractability of density estimation in high-dimensional spaces, we employ a nonparametric particle-based entropy estimator to quantify the spatial diversity of UAV trajectories. This entropy estimate is utilized as an intrinsic reward, incentivizing agents to maximize the distance between their trajectories and those of their neighbors. Extensive simulations demonstrate that AEQMIX significantly outperforms baseline reinforcement learning and traditional optimization methods in terms of search rate, coverage efficiency, and collision avoidance. Compared with DNQMIX, AEQMIX improves the search rate and coverage rate by 9.52% and 11.54%, respectively, while reducing the average collision count by 70.59% in the (40 × 40) environment.
