Exploring LLM Capabilities for Situational Understanding and COLREG compliance on real-world maritime navigation scenarios

CAPTAIMN: A Real-Time LLM and RAG Based Decision Support System for Navigational Safety and COLREGs Compliance

The correct regulatory interpretation in naval environments is challenging due to the complexity and urgency of decisions based on the International Regulations for Preventing Collisions at Sea (COLREGs). This article presents the development of an intelligent agent named Cognitive Agent for Analysis of Interrelated Problems in Maritime Navigation, hereafter referred to as CAPTAIMN. This agent integrates Large Language Models (LLM) and a Retrieval-Augmented Generation (RAG) architecture to support human decision-making and officer training in safety-critical naval environments. Thus, this work aims to propose a methodology for building an intelligent agent based on LLM and RAG, specifically focused on the assisted and contextualized interpretation of COLREGs. The proposed methodology was evaluated through a quantitative study with 15 maneuvering officers, who assessed 150 responses generated by a local language model using a Likert scale. The results from this phase showed significant approval, with 80% of the responses being rated as 'Agree' or 'Totally Agree' by the officers. These results suggest that the integration of LLM and RAG through CAPTAIMN can provide useful support for both decision-making and tactical training in naval operations.

Can Large Language Models Explain Flight Safety Events? A Prior-Guided Semantic LLM-based Approach

Improving flight safety with flight data requires not only accurate detection of risk events, but more importantly, clear interpretation of their underlying causes at the level of pilot control behavior. Existing explainable AI techniques, such as feature importance maps, often require considerable domain knowledge to translate them into operationally meaningful explanations. Large Language Models (LLMs), which excel at language reasoning, bring a promising solution to this issue. However, applying LLMs in this domain presents key challenges such as modal inconsistency, limited classification ability, scarcity of task-specific data for fine-tuning, and lack of domain knowledge. To overcome these challenges, we propose FlightLLM, a prior-guided semantic LLM-based approach for interpretable flight safety analysis. Specifically, we first perform feature engineering to address modal inconsistency, combining statistical descriptors with physically meaningful flight indicators. This representation is further processed by a Semantic Discretization module, which converts abstract numerical patterns into qualitative descriptions that are more compatible with language reasoning. In addition, since LLMs are not inherently strong classifiers, CatBoost is incorporated as a statistical expert, and its prediction results are injected into the prompt as prior guidance. A contrastive few-shot learning strategy is further adopted to compensate for limited data. Finally, we design structured prompts to embed aviation-specific knowledge into the inference process. Using hard landing, a representative risk event with complex causal mechanisms, as an anchor point, we evaluate FlightLLM on a dataset of 704 real-world A320 flight samples. Experimental results show that the proposed approach achieves competitive classification performance while generating direct and reasonable explanations for event causes.

Lang2Graph: Towards Leveraging Human Language for Indoor Topology Inference Using LLMs

Edge-deployed systems such as autonomous robots, AR/XR devices, and emergency-response handhelds require accurate indoor topological representations, yet existing sensor-based and expert-curated mapping methods are impractical for crowd-sourced, resource-constrained deployment. Additionally, current large language model (LLM) approaches to indoor topology inference lack a systematic framework for evaluating the factors that govern performance. We present Lang2Graph, an experimental framework for indoor topological graph inference from natural-language navigational instructions that isolates four governing factors: instruction structure, metadata clarity, prompting strategy, and model size and reasoning capability. We propose the Independent Prompt Executor (IPE), a prompting strategy that decomposes graph construction into independent per-instruction reasoning steps, preventing error propagation. To support factor-level evaluation, we develop a fully synthetic dataset and an augmented benchmark (R2R-AUG) covering idealized and real-world conditions. Our evaluation across multiple model families show that structured instructions, clear metadata, and IPE improve precision, recall, and F1 by 44%, 45%, and 47%, respectively. Reasoning-aligned open-source models of moderate scale (14B parameters) outperform larger proprietary models on the most challenging instruction categories, indicating that reliable indoor topology inference is achievable without cloud-scale dependencies and establishes a viable path toward on-device edge deployment.

AeroGround: A Comprehensive Benchmark for Aerial-Ground Collaborative Reasoning

Vision-language models (VLMs) have been widely employed in understanding and reasoning tasks for unmanned aerial vehicles (UAVs). Existing UAV benchmarks primarily focus on aerial-view scenarios. However, whether current VLMs can perform well on understanding and reasoning tasks in aerial-ground collaborative scenarios which are practical in real-world applications like rescue and infrastructure inspection remains underexplored. To address this gap, we introduce AeroGround, a comprehensive benchmark for evaluating VLMs in aerial-ground collaborative reasoning. AeroGround is built upon a simulated aerial-ground dataset containing approximately 29,000 multimodal observation groups from diverse open environments, and provides 2,250 high-quality question-answering instances covering cross-view correspondence, spatial understanding, and reasoning. Experiments on 16 pretrained VLMs, together with two domain-adapted variants, reveal a substantial gap between current models and human performance: the best model achieves an average accuracy of 54.4%, whereas humans reach 93.3%. By systematically revealing the strengths and limitations of existing models in aerial-ground collaborative reasoning, AeroGround provides a foundation for developing more capable aerial-ground collaborative embodied intelligence systems.

NavAI: an application-agnostic llm framework for navigation tasks in virtual reality environments

LAPF: LLM-Agent-Based Path Finder Using the UAVScenes Dataset

Uncrewed aerial vehicles (UAVs) are increasingly deployed for autonomous navigation in complex outdoor environments, where dynamic conditions and mission requirements require intelligent adaptive decision-making. Existing optimization-based, Machine Learning (ML), and Reinforcement Learning (RL) approaches often rely on predefined models or task-specific training, limiting their generalization and adaptability in uncertain scenarios. Recent Large Language Model (LLM)-assisted approaches offer promising reasoning capabilities but remain constrained by limited agentic functionality, including insufficient memory, planning, and tool interaction mechanisms.This paper proposes an LLM-Agent-Based Path Finder (LAPF) framework for autonomous UAV navigation in town-scale outdoor environments. LAPF extends LLM-assisted navigation by integrating perception, memory, planning, and action modules into a closed-loop cognitive architecture. The proposed agent leverages prior navigation experiences, performs Chain-of-Thought (CoT) reasoning, couples each detected hazard to a bounded corrective action, and dynamically refines waypoint decisions based on environmental feedback.The three independent trials per method demonstrate that LAPF achieves mean path lengths of 512.83 m and 506.37 m, compared to the straight-line optimum of 497.33 m, corresponding to path length reductions of 17.2% and 15.6% relative to CoT prompting and absolute path efficiencies of 97.1% and 98.1% in open-field and obstacle-injected scenarios, respectively. Furthermore, LAPF is the only evaluated approach that couples every detected hazard to a bounded, metric-neutral corrective action while maintaining near-goal stability, with zero clamp events in both scenarios, whereas CoT prompting increases from 9.7 to 14.0 events.