RecoverFly is proposed, a failure-aware RL post-training framework for end-to-end UAV-VLA policies that adapts token-level RL for stable optimization of grammar-constrained autoregressive UAV actions, revisits unresolved failure cases to strengthen corrective learning and sample utilization, and combines a two-stage long-tail scene curriculum with reference-policy regularization to improve scene adaptation while preserving acquired capabilities.
Abstract
Unmanned aerial vehicle vision-language navigation (UAV-VLN) requires agents to translate visual observations and language instructions into reliable flight actions in complex environments. Although recent end-to-end UAV vision-language-action (UAV-VLA) policies reduce reliance on separately designed perception, planning, and control modules, their behavior-cloning objectives provide limited corrective supervision for interactive closed-loop execution. Reinforcement learning (RL) offers a promising solution, while its effectiveness is constrained by inefficient use of samples, long-tailed scene distributions, and policy distribution shift during optimization. To this end, we propose RecoverFly, a failure-aware RL post-training framework for end-to-end UAV-VLA policies. Specifically, RecoverFly adapts token-level RL for stable optimization of grammar-constrained autoregressive UAV actions, revisits unresolved failure cases to strengthen corrective learning and sample utilization, and combines a two-stage long-tail scene curriculum with reference-policy regularization to improve scene adaptation while preserving acquired capabilities. Experiments on the TravelUAV benchmark demonstrate that RecoverFly achieves the best performance on the seen, unseen-map, and unseen-object splits. Moreover, compared to the AerialVLA initialization, RecoverFly improves success rate by 3.12 to 8.37 percentage points under a total rollout budget of about 30\% of the training-set size, validating its effectiveness, robustness, and generalization capabilities.
Autonomous navigation of low-altitude unmanned aerial vehicles (UAVs) in cluttered environments is challenging due to partial observability, limited onboard perception, and inefficient exploration in end-to-end reinforcement learning. This paper proposes a teacher-guided asymmetric reinforcement learning framework for end-to-end visual navigation of low-altitude UAVs in the Isaac Sim 5.1 environment. A privileged teacher policy is first trained using obstacle-state information to acquire reliable navigation priors. A deployable student policy is then learned with an asymmetric actor-critic architecture, where the actor takes depth images and proprioceptive states as input, while the critic uses privileged information during training. To improve policy transfer, an annealed knowledge distillation strategy is adopted: the student is strongly guided by the teacher in the early stage, and the guidance is gradually removed to enable autonomous reinforcement refinement. Experimental results show that the proposed method achieves higher success rates, lower collision rates, and faster convergence than baseline methods.
Yi-Min Wei, Qiu-Quan Guo, Cai-Zheng Wang et al.· 2026 IEEE International Conf...· 0 citations
Autonomous navigation in cluttered environments is hampered by partial observability and dynamic constraints. This paper presents PILOT, a constraint-aware privileged imitation learning framework for vision-based end-to-end UAV motion planning under partial observability. The framework distills planning strategies from a computationally intensive optimal control expert into a student policy regularized toward safety and dynamic requirements via a dual-objective loss function. To mitigate partial observability, a spatiotemporal perception fusion module using a Temporal Convolutional Network (TCN) is developed to integrate historical depth images and odometry. This module infers task-relevant latent context from historical observations, enhancing spatial awareness beyond the instantaneous FOV without maintaining persistent map memory. A trajectory parameterization layer mapping network outputs to a structured trajectory, while enabling explicit continuity, dynamic-consistency, and obstacle soft penalties during training, encouraging constraint satisfaction for unseen observations without formal guarantees. Simulations on quadrotor and fixed-wing aircraft demonstrate that PILOT achieves performance comparable to the privileged expert while reducing computational overhead by over 80\%. Successful indoor and outdoor zero-shot deployment confirms the practical feasibility and cross-domain generalization of the planner.
Qing-Rui Zhang, Feng Xue, Xiang Zhou et al.· 0 citations
Coordinating multiple unmanned aerial vehicles (UAVs) for cooperative missions requires agents that perceive their environment, reason about objectives, and generate joint actions. Vision–language–action (VLA) models unify these capabilities but lack a principled source of multi-agent training data and suffer from a training–inference discrepancy in closed-loop control. We propose MiniUAV-VLA, a compact centralized VLA controller for simulated multi-UAV search-and-elimination based on multi-agent reinforcement learning (MARL) expert distillation. A QMIX expert policy achieving 100% mission success generates multimodal demonstrations pairing rendered tactical map images with structured textual state prompts. A 158 M-parameter VLA model with approximately 65 M trainable parameters in the MiniMind-3V backbone and vision projection is fine-tuned with a multi-agent discrete action head that jointly predicts actions for all UAVs in a single forward pass. We identify a training–inference feature mismatch in behavior cloning and address it via prompt-end action pooling, which extracts action-relevant hidden states at the user–prompt boundary rather than after the generated response. In closed-loop evaluation with four drones and six mobile targets averaged over five evaluation seeds, MiniUAV-VLA reaches 74.4 ± 4.6% mission success against 9.4 ± 2.1% for a random policy and 16.2 ± 3.2% for an observation-limited greedy baseline. Across five independent training runs, prompt-end action pooling improves mean closed-loop success from 40.6% to 76.2% over the last-token alternative. These results support MARL expert distillation as a data-efficient route to compact multi-agent VLA control in this simulated setting.
Hongwei Han, Guanghong Gong, Ni Li· Drones· 0 citations
UAV vision-language navigation (UAV-VLN) focuses on enabling an aerial agent to follow natural-language instructions in open 3D environments from egocentric visual observations. Current approaches suffer from three coupled issues: weak grounding of instruction-relevant landmarks in visual observations, insufficient exploitation of long-horizon history, and unstable decisions under local traps or repeated exploration. To address these issues, we propose a unified semantic-to-decision framework. First, we present an instruction-grounded semantic enhancement module that injects object-level semantics and relative spatial cues into the current observation state. Subsequently, we develop a relevance-aware dynamic temporal aggregation strategy that reweights the full history buffer while converting a few high-relevance frames into structured landmark prompts for the decoder. Finally, we devise a topology-aware decision method that combines local-optimum cognition with group-relative policy optimization under progress, goal, semantic, and path-compliance rewards. Experiments on the widely used AerialVLN and OpenFly benchmarks clearly demonstrate that our method achieves state-of-the-art performance.
This work provides a feasible technical pathway and reproducible evaluation benchmark for the collaborative deployment of lightweight LLM planner, sub-goal guidance, sensor observations, cooperative reward, and reward shaping components and quantifies the indispensability of the LLM planner.
Yuting Cao, Zheng Zhao, Jiekai Wu et al.· Journal of King Saud Univers...· 0 citations
LAPF is the only evaluated approach that couples every detected hazard to a bounded, metric-neutral corrective action while maintaining near-goal stability, with zero clamp events in both scenarios, whereas CoT prompting increases from 9.7 to 14.0 events.
Yousef Emami, MohammadHossein Homaei, Hao Zhou et al.· 0 citations
Known for his clear and elegant writing style, Bertsekas shaped fields from control and optimization to large-scale computation and artificial intelligence.
