Skip to content

NavAI: an application-agnostic llm framework for navigation tasks in virtual reality environments

Aug 2026 · International Conference on Automated Software Engineering · Vol 33 · 0 citations · 46 references

TL;DR

NavAI is an extensible navigation framework that leverages large language models (LLMs) to support both basic action commands and multi-step goal-oriented navigation through an application-agnostic screenshot-and-control interface and explores optimization strategies for virtual scene understanding and navigation goal decision making.

View source

Similar papers

Open access Aug 2026

DevGRU: Depth-Guided Visual Navigation Using a Collision-Aware Recurrent Model

The proposed DevGRU navigation system employs an action predictor that generates collision-aware future trajectories, enabling effective avoidance of immediate obstacles and has a relatively small number of trainable parameters, resulting in the fastest inference time among the baselines.

Kyung Min Han, Eunsom Kim, Young J. Kim · 0 citations
Conference Open access Jul 2026

ZONDA: Zero-shot Object Navigation with Dynamic Avoidance in Multi-floor Environments

ZONDA, a zero-shot object navigation with dynamic avoidance framework, integrates three core components and can maintain robust navigation on the dynamic benchmark HM3D-DYNA compared to the existing baseline.

Shao-Min Liang, Xuan-Hong Liao, Shi-Yao Zhang · 0 citations
Preprint Aug 2026

Embodied-Navigator: Point, Think, Memorize, and Align for Efficient Navigation

This work proposes TAMP-Nav, a unified framework for efficient embodied navigation that dynamically triggers Chain-of-Thought and retains high-fidelity memory only at critical nodes, compressing redundant trajectories into lightweight Space-Time Indicators, thereby preserving critical historical information and enhanci...

Hongyan Feng, Sun-Lai Chen, Xuan-Yu Liu et al. · 1 citation
Jul 2026

SoftNav: Injecting 3D Scene Tokens into VLMs for Embodied Navigation

SoftNav is introduced, which injects entity-level 3D continuous representations -- one token per detected object or frontier -- into a VLM's hidden space as soft tokens through a lightweight projector, enabling transferable navigation with minimal training.

Yi Wu, Junjie An, Xiao Liu et al. · 0 citations
Conference Open access Sep 2026

FILD-Nav:Vision-and-Language Navigation with Instruction Landmark Features in Continuous Environments

FILD-Nav extracts task-relevant landmarks from instructions and incorporates landmark semantics into both waypoint prediction and topological planning, which improves waypoint relevance and enables more effective long-horizon navigation.

Chuan-Ye Hu, Lu-Lu Liu, Huai-Wei Si et al. · 0 citations

Related blog posts

Microsoft Research Blog Oct 12, 2026

Simulating student learning for better AI tutoring

StudentSim is a research system for evaluating and improving AI tutoring tools before they are tested with learners. It models how individuals may engage with tutors, giving feedback to improve tutor performance across subjects like chess, writing, and mathematics. The post Simulating student learning for better AI tutoring appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.