End-to-End LLM Flight Planning with RAG-based Memory and Multi-modal Coach Agent

SLM-A*: Compact Language-Guided Autonomous Agentic Planner for Risk-Aware Multi-UAV Path Planning

Multi-UAV path planning in threat-aware environments requires generating trajectories that reach mission objectives while maintaining obstacle clearance, inter-agent separation, and computational efficiency. This paper presents SLM-A*, a hybrid language-to-waypoints planning framework that fine-tunes compact transformer models to generate discrete multi-UAV waypoint sequences from structured mission descriptions. The generated trajectories are treated as candidate plans and are subsequently evaluated using a deterministic geometric safety checker that verifies obstacle avoidance, risk exposure, inter-UAV separation, edge-level conflicts, and motion feasibility. We construct a dataset of 3,000 static threat-aware scenarios consisting of three UAVs, four threats, one objective, and a $40\times 40$ grid using a prioritized time-expanded A* planner with clearance and turn penalties. Six small language models, including Gemma-2B, Fox-1-1.6B, Qwen2.5-1.5B, Phi-2-2.7B, StableLM-3B, and TinyLlama-1.1B, are adapted using QLoRA with 4-bit NF4 quantization. Among these models, TinyLlama-1.1B achieves the strongest overall text-level performance and inference throughput and is selected as the SLM-A* planning module. On a 315-scenario held-out benchmark, SLM-A* achieves the best average values on four of five evaluated trajectory metrics, including makespan, sum of costs, conflict count, and risk exposure, while CBS maintains the largest minimum-separation margin. Gazebo-ArduPilot-MAVROS simulations further demonstrate that the generated waypoint sequences can be executed by a realistic UAV control stack. The results suggest that compact language models, when combined with explicit geometric validation, can serve as efficient candidate planners for multi-UAV waypoint generation under resource-constrained deployment settings. Beyond trajectory performance, SLM-A* is designed as a language-native planning agent that can interoperate with multi-agent LLM frameworks as a callable planning sub-module, thus providing an architectural capability not available to conventional planners that operate on numerical graph representations.

Real-Time Multi-Robot Motion Planning with Safe-Interval Search and Learning-Guided Repair

No Training, Better Flights: Test-Time Scaled VLMs for UAV Navigation

Test-time scaling offers a promising method to improve the inference performance of Vision-Language Models (VLMs) without additional training. Existing approaches to vision-language navigation (VLN) for Unmanned Aerial Vehicle (UAV) typically relies on a single inference pass, which can falter in complex environments by producing suboptimal or unsafe trajectories. In this paper, we explore a simple and effective approach to apply test-time scaling to VLN for UAV. We enhance navigation reasoning through an iterative refinement process that requires no extra model training, guiding the model to re-evaluate its initial navigation plan for better accuracy and safety. Our method first prompts the model to generate multiple parallel candidates and then performs a self-correction step, achieving deeper and more robust planning without changing the underlying model. To further strengthen decision-making, we design a multi-criteria scoring function to evaluate the refined candidates based on safety, goal alignment, and forward-progress. This simple yet powerful combination enables a frozen UAV navigation VLMs to self-correct and generate more accurate and reliable flight plans, achieving SOTA performance in this task.

Search-Aided Joint Agent-Environment Reinforcement Learning for Robust Lifelong Multi-Agent Path Finding with Rotations

Lifelong Multi-Agent Path Finding (LMAPF) requires repeatedly planning collision-free paths for agents that continuously receive new goals upon reaching their current ones. While many learning-based planners have been proposed for LMAPF, most rely on oversimplified kinematic assumptions that may overlook motion constraints critical to real-world performance. In this work, we study a more realistic LMAPF model derived from many real-world automated warehouse systems, termed LMAPF-R2, which incorporates robust safety constraints and in-place rotation constraints. These constraints substantially increase coordination difficulty, particularly in highly constrained spaces. To address these challenges, we propose Search-Aided Joint Reinforcement Learning (SJRL). We first augment neural policies with Causal PIBT, a single-step search-based planner that resolves agents'collisions and propagates their intentions. We then introduce a unified RL formulation that jointly optimizes agent and environment policies, where the environment policy learns graph edge costs to provide global movement guidance via backward Dijkstra search. Experiments demonstrate that SJRL achieves significant improvements over the strong search-based planner, Causal-PIBT, across multiple high-density maps. We further validate SJRL in a challenging mixed-reality warehouse environment with 8 physical robots and 248 virtual robots.

CoMuRoS - An LLM-based generalizable hierarchical task planning and execution framework for heterogeneous robot teams with event-driven re-planning

Heterogeneous multi-robot teams require systems that can interpret natural-language goals, allocate tasks, and adapt to unexpected events. We developed CoMuRoS (Collaborative Multi-Robot System), a generalizable hierarchical architecture combining a centralized task-manager LLM with decentralized robot-level LLMs for executable Python generation from primitive ROS2 skills. The task manager uses static planning rules and dynamic context, including task history, robot/task status, and detected events, while onboard perception using VLM/image processing classifies events as relevant or irrelevant and triggers replanning. Hardware experiments demonstrated recovery from disruptive events, filtering of irrelevant distractions, and coordinated transport with emergent human-robot cooperation, achieving success rates of 9/10 for collaborative object recovery, 8/8 for coordinated transport, and 5/5 for human-assisted recovery. Simulation studies demonstrated intention-aware replanning. A curated benchmark of 22 scenarios, 54 tasks, and around 20 robots evaluated task allocation, classification, IoU, executability, and correctness across multiple LLMs, with correctness up to 0.91 ± 0.053; a 20-scenario replanning benchmark achieved Correctness = 0.948 ± 0.034 using Grok 3. CoMuRoS enables runtime, event-driven replanning on physical robots and supports flexible multi-robot and human-robot collaboration across diverse scenarios.

World-Model-Grounded LLM Planning for AUV and ASV Navigation Near Offshore Wind Farms

Large language models can turn a natural-language mission into a sequence of robot actions, but they do not have a sense of physics: they cannot judge how long a command should run, or whether it will make the robot drift into an obstacle. We proposed the use of a world model to expand the capabilities of Large Language model-based planners. Our method has three components: a physics-grounded neural world model, a three-phase gradient-based trajectory optimizer, and a Model Predictive Controller (MPC)-style closed-loop replanner with a trust-region guard. The language model decides what to do, and the world model decides how long, whether that means driving eight thrusters through 6 DOF or two differential thrusters through 3 DOF. We evaluate two marine vehicle classes operating near offshore wind infrastructure: a 6-DOF Autonomous Underwater Vehicle (AUV) and a 3-DOF differential-drive Autonomous Surface Vehicle (ASV). In five benchmark missions per platform, both vehicles reach every goal with zero predicted collisions, and both transfer to GazeboSim under ocean current, waves, and thruster dynamics, remaining collision-free and cutting GazeboSim goal-distance error versus the ungrounded baseline by 70-82% (ASV) and roughly 93% (AUV), after a residual fine-tuning pass that separately reduces surrogate rollout Root Mean Square Error (RMSE) by 60% (AUV) and 69% (ASV). For the ASV we further demonstrate a Vision language model (VLM)-assisted semantic-mapping pipeline that extracts obstacles and environmental context from satellite imagery, nautical charts, and forecast Application Programming Interface (API) instead of onboard sensors, reaching 96% navigability accuracy as a drop-in replacement for hand-specified obstacle geometry.