Jul 2026· 2026 2nd International Conference on Unmanned Systems and Technology (UST)· pp. 6-10· 0 citations· 15 references
Abstract
Addressing the challenges of real-time decision-making and environmental adaptability in multi-unmanned systems, this study introduces a Transformer-based dynamic graph attention mechanism applied to multi-UAV coordinated navigation tasks. The approach first employs Graph Neural Networks to encode UAV positions, velocities, and environmental obstacles, extracting spatio-temporal node features. A Transformer encoder then processes temporal sequences to generate global contextual representations. A dynamic fusion layer follows, using task goal semantic embeddings as queries to adaptively adjust attention weight distributions, mitigating performance degradation in uncertain environments caused by static models. Subsequently, a multi-layer perceptron decoder outputs optimized collaborative paths, culminating in end-to-end planning via a policy network. During training, reinforcement learning is combined with imitation learning auxiliary losses to enhance model robustness and generalization. This methodology effectively improves path planning efficiency and safety for multi-unmanned systems in complex dynamic scenarios. Experimental evaluations across various environments demonstrate superior success rates, reduced path lengths, and improved computational efficiency compared to baseline methods such as GNN, RL, and standard Transformers. The proposed model achieves an average success rate of over 91%, underscoring its potential for practical deployment in autonomous navigation systems.
With the increasing deployment of multi-unmanned aerial vehicle (multi-UAV) systems in dynamic environments, the problem of efficient cooperative path planning has emerged as a critical challenge requiring urgent solutions. To address this issue, this paper proposes a novel joint optimization framework, named spatio-temporal attention-based multi-agent deep deterministic policy gradient (STA-MADDPG). Rather than proposing a new reinforcement learning algorithm in the strict sense, this work integrates advanced spatial-temporal feature extraction with heuristic gradient guidance. First, a cascaded architecture combining multi-head attention and Long Short-Term Memory (LSTM) networks is utilized to extract key local and temporal features, thereby mitigating the dimensionality curse in dense multi-agent observations. Second, an improved dynamic artificial potential field (DAPF) is integrated into the reinforcement learning framework as a state augmentation mechanism, providing heuristic guidance vectors that accelerate convergence and improve obstacle avoidance. Furthermore, to balance computational complexity and adaptive behavior, a rule-based hierarchical formation strategy is designed. The framework maps predefined formations (elliptical, chain, or wedge) to specific environment categories, while the underlying MARL policy governs the dynamic trajectory planning and topology maintenance. Finally, rigorous comparative and ablation experiments are conducted to evaluate path length, search time, and relative position errors. Statistical analysis demonstrates the effectiveness of the proposed framework, achieving up to a 67.3% reduction in search time and a 91.56% search success rate compared with standard MARL baselines in complex environments.
Robot path planning in dynamic environments is a critical research domain in autonomous robotics, focusing on safe and efficient navigation under uncertain and continuously changing conditions. The presence of moving obstacles, unpredictable environmental variations, and real-time decision-making constraints makes traditional path planning methods less effective. To address these challenges, this study proposes a novel Transformer-guided Meta-learning and Graph-enhanced Multi-Agent Deep Reinforcement Learning (TMG-MADRL) framework for intelligent robot navigation. The framework integrates Vision Transformer (ViT) for extracting global environmental and obstacle features, Graph Attention Network (GAT) for spatial relationship modelling, Model-Agnostic Meta-Learning (MAML) for rapid adaptation to unseen environments, Multi-Agent Proximal Policy Optimization (MAPPO) and Soft Actor-Critic (SAC) for cooperative policy optimization, and Temporal Graph Transformer Network (TGTN) for future obstacle trajectory prediction. The work aims to achieve robust path optimization, adaptive decision-making, and proactive collision avoidance in dynamic scenarios. The framework is trained and evaluated using the Robot Path Planning Navigation Dataset, where navigation states are classified into Collision and Safe categories. Experimental results achieved 98.24% accuracy, 98.17% precision, 98.42% recall, 98.06% specificity, and 97.91% MCC, validating superior navigation efficiency and collision classification performance.
Shu-Lin Song, Lan Wu· Journal of engineering and a...· 0 citations
In complex and unknown environments, unmanned aerial vehicle (UAV) autonomous navigation still faces issues such as insufficient representation of state characteristics, fixed reward guidance, and low efficiency in utilizing key experience samples. To address these problems, this paper proposes an improved soft actor–critic (SAC) method that integrates dual-path channel attention (DPCA), adaptive reward feedback (ARF), and prioritized experience replay (PER); this method is named DPCA-ARF-PER-SAC. The proposed DPCA module is introduced into the actor network to recalibrate one-dimensional navigation state features and enhance the representation ability of key decision-making information. At the same time, the ARF mechanism can dynamically adjust the reward weights according to the training progress, while PER is used to improve the utilization efficiency of key samples. The experiments are conducted in a two-stage structure, including module-level ablation verification in the two-dimensional (2D) SimpleAvoid scenario and main performance comparison in the three-dimensional (3D) NH_center scenario. The experimental results show that the success rate of this method reaches 1.00 in the 2D scenario, and the collision rate is 0.00. In the 3D scenario, the success rate is 0.76, the collision rate is 0.24, and the average episode length is 232.8 steps. Compared with the baseline SAC, the success rate is increased by 2 percentage points, the collision rate is reduced by 2 percentage points, and the average episode length is reduced by 4.3 steps. Compared with twin delayed deep deterministic policy gradient (TD3), SAC, and representative reinforcement learning methods such as attention-mechanism SAC (AM-SAC) based on attention mechanism enhancement, the proposed DPCA-ARF-PER-SAC method shows more balanced performance in task completion, navigation safety, and path efficiency. These results indicate that DPCA-ARF-PER-SAC provides a more robust navigation strategy for complex 3D UAV autonomous navigation tasks.
Yufei Wang, Tong Zhang, Fan Zhou et al.· Applied Sciences· 0 citations
A multi-agent deep reinforcement learning framework that addresses issues through coordinated exploration, demonstration exploitation, safe curriculum scheduling, and structure-aware generalisation is proposed, demonstrating strong performance in collaboration success rate, navigation robustness, zero-shot cross-scenario generalisation, and dynamic environment adaptability.
An Attention-based Multi-Agent Deep Deterministic Policy Gradient algorithm was developed for cooperative multi-unmanned aerial vehicle target tracking in dynamic environments. The study addressed information redundancy and association weight allocation between individual unmanned aerial vehicles and the swarm during cooperative decision-making. To improve information selection, the proposed algorithm introduced a centralized critic network with a multi-head attention mechanism to evaluate the contributions of other agents at each time step. Meanwhile, the study designed a dual-buffer experience management architecture composed of a recent interaction memory and a mission outcome memory. This architecture stored recent interaction data and mission-critical trajectories separately, thereby improving experience utilization during training. The study also formulated the state space, action space, and reward function for target exploration, obstacle avoidance, energy consumption, and velocity maintenance under limited perception conditions. The experiments evaluated the proposed algorithm in a two-dimensional tracking scenario with moving targets, multiple unmanned aerial vehicles, and dynamic and static obstacles. The proposed method was compared with Deep Deterministic Policy Gradient and Multi-Agent Deep Deterministic Policy Gradient using collision rate, capture rate, capture time, and capture distance as evaluation metrics. The results showed that the proposed algorithm improved convergence behavior and average reward, reduced the collision rate by 83% compared with Deep Deterministic Policy Gradient at the first environmental level, and maintained competitive performance in capture rate, capture time, and path efficiency.
Qinglin Han, Hongmei Wang· International Conference on...· 0 citations
Ensuring QoS provisioning in low-altitude wireless networks requires UAV positioning and navigation strategies that adapt to dynamic environments and generalizes across heterogeneous network scenarios. This paper proposes a digital twin (DT)-assisted meta reinforcement learning framework for multi-agent UAV trajectory planning. A high-fidelity network DT serves as a supervisory layer to generate key performance indicators (KPIs) and fine-grained channel knowledge, which guides both domain-specific learning and cross-domain validation. Building on the twin-informed UAV landmarks, we then develop a weakness-aware meta learning scheme: in the inner loop, agents are trained cooperatively toward the self-discovered landmarks under dynamic conditions; in the outer loop, navigation policies are evaluated via the DT to identify bottlenecks and generate targeted hard scenarios, enabling robust adaptation across diverse scenarios. Extensive simulations show that our framework achieves up to 4× higher service coverage compared to baselines, while the target-aware outer-loop adaptation further improves cross-scene performance and model generalization.
Jiayuan Huang, E. Tucker, Ruozhou Yu et al.· International Conference on...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
