Executable Reference Trajectory Construction and Conflict-Aware Residual Reinforcement Learning for Urban Multi-UAV Navigation

Abstract

Urban multi-UAV navigation in dense building environments requires not only collision-free geometric paths but also executable flight processes under motion constraints and inter-UAV safety requirements. A static path that is feasible in a geometric map may still fail during closed-loop execution because of velocity limits, acceleration constraints, local path-association errors, and coupled multi-UAV interactions. Meanwhile, end-to-end reinforcement learning often suffers from unstable training, weak geometric interpretability, poor early-stage safety, and high sample complexity. To address these issues, this paper proposes a hierarchical planning-and-learning framework that connects static reference path generation, executable reference tracking, successful demonstration distillation, and conflict-aware residual reinforcement learning. First, three-dimensional reference paths are generated offline in an OpenStreetMap-based urban scene represented by cuboid buildings. Second, a damped reference-tracking mechanism transforms these static paths into closed-loop executable reference processes through local path association, monotonic progress updating, path recapture, look-ahead guidance, and bounded action construction. Third, successful pure-reference executions are distilled for behavior-cloning initialization. Finally, a bounded residual TD3 module is introduced as a local conflict-correction mechanism around the verified executable reference baseline. Experiments in an urban scene containing 754 buildings show that simplified tracking strategies fail to execute the static paths reliably, whereas the proposed full-damped reference-tracking controller achieves a 91.67% all-success rate and eliminates building collision episodes in the tracking-ablation test. Speed-sensitivity experiments at 10, 15, and 20 m/s show the same 91.67% all-success rate, indicating that the conclusion is not dependent on a single speed setting. In constructed conflict-stress tests, the conflict-aware residual TD3 module increases the all-success rate from 33.33% to 80.09%, reduces inter-UAV collision episodes from 66.67% to 11.57%, and improves the hard-safety satisfaction rate from 33.33% to 87.04%. These results show that the main contribution of the proposed framework lies in converting static geometric paths into executable reference trajectories and further enabling bounded residual correction under inter-UAV conflict conditions.