Skip to content
Preprint

Asking Less, Adapting Better: Feedback-Efficient Test-Time Adaptation for Vision-and-Language Navigation

Oct 2026 · 0 citations · 52 references
Computer Science

TL;DR

This work proposes FETA, a Feedback-Efficient Test-Time Adaptation method that improves both feedback efficiency and optimization reliability, and introduces a hierarchical sample selection strategy to localize informative samples.

Abstract

Vision-and-Language Navigation (VLN) requires agents to follow instructions and navigate to target locations. However, current VLN agents often suffer severe performance degradation in unseen environments due to distribution shifts. Test-Time Adaptation (TTA) offers a promising paradigm to bridge this gap, yet existing methods that rely on unsupervised entropy minimization can propagate errors through the trajectory, causing cascading failures in navigation. Recent methods mitigate this by incorporating human feedback, but they apply trajectory-level supervision uniformly across all steps without localizing the actions is truly informative, leading to inefficient feedback utilization and mismatched optimization granularity. To address these limitations, we propose FETA, a Feedback-Efficient Test-Time Adaptation method that improves both feedback efficiency and optimization reliability. To reduce unnecessary human interventions, we introduce a hierarchical sample selection strategy to localize informative samples. Specifically, we first filter trajectories whose normalized policy entropy exhibits large fluctuations. Within each selected trajectory, we further select the critical action via an entropy-gradient saliency criterion and query a single binary feedback on its correctness. A discrepancy-aware entropy optimization objective then updates the policy only at the queried step, with adaptive scaling that assigns larger relative weights to samples with greater prediction-feedback discrepancy. Extensive experiments across multiple benchmarks demonstrate that FETA achieves state-of-the-art performance with fewer human interventions, while remaining robust under noisy feedback and continual adaptation settings.

View source

Similar papers

Credit-Guided Policy Improvement for Test-time Adaptive Vision-Language Navigation

Credit-Guided Policy Improvement (CGPI), which recovers signed, reference-relative decision credit from action-induced observation transitions without external outcome feedback from action-induced observation transitions without external outcome feedback is proposed.

Yang Li, Si-Jia Zhang, Yi-Han Li et al. · 0 citations

AVERT-VLN: Abstention-aware Visual Error Recovery and Training for Vision-and-Language Navigation

Deploying vision-and-language navigation (VLN) agents in unseen environments remains challenging because unfamiliar layouts and visual conditions can cause execution to go off track. Rather than relying on continuous human supervision, a practical strategy is to selectively request corrective guidance, recover the ongo...

Min-Rui Liu, Jing-Ke Wang, Yue-Hao Huang et al. · 1 citation
Preprint Nov 2026

Remember, Replay, Adapt: Learning from Past Experience in Online Vision-and-Language Navigation

Existing feedback-based methods for online vision-and-language navigation (VLN) primarily learn from current episode outcomes, leaving previously acquired feedback underutilized. Inspired by human episodic memory, we propose R2A, an online VLN adaptation framework that continually reuses successful and failed navigatio...

Chen-Hao Ding, Ji-Zhou Han, Li-Xin Xiu et al. · 0 citations
Preprint Nov 2026

SURE:Spatiotemporal Uncertainty Representation for Efficient Vision-Language-Action Learning

Vision-language-action (VLA) policies have become a promising paradigm for end-to-end robot control, but their effectiveness still depends heavily on large volumes of demonstration data. In manipulation tasks, this dependence is compounded by structured spatiotemporal uncertainty in the action space: multiple nearby ac...

Jing-Ye Zhang, Choo Sin Wai, Da-Ke Zhong et al. · 1 citation
#artificial intelligence Preprint Oct 2026

VLA-ZO: Fast Zeroth-Order Adaptation for Vision-Language-Action Models

Adapting vision-language-action (VLA) models to deployment-time distribution shifts is important for reliable robotic operation, but conventional first-order adaptation can exceed the memory budget of inference-oriented deployment platforms. Zeroth-order (ZO) optimization offers a forward-only alternative with inferenc...

Jaemin Kim, Jiahn Kim, Taesik Gong · 0 citations
Conference Open access Oct 2026

Aligning Flow-Matching Vision-Language-Action Models via Advantage-Guided Action Preference Optimization

Deploying robust manipulation policies on robotic arms with lightweight Vision-Language-Action (VLA) models remains challenging. Teleoperated demonstrations often contain human hesitation and suboptimal corrections, which Supervised Fine-Tuning (SFT) inadvertently mimics rather than learning decisive control. In this p...

Yan Shen, Qing-Shan Liu · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.