Vision-based Navigation Models for Autonomous Rovers: Experimental Analysis and Comparison

Abstract

Traditional navigation algorithms rely heavily on the fusion of heterogeneous data from different sensor modalities for precise localisation and navigation tasks. However, these approaches lack generalisation and interaction with human operators, particularly in navigating unknown environments and human language-based task specifications. In contrast, the foundation models trained on internet-scale data proved higher generalisation capabilities and show an emerging trend in zero-shot learning for navigation tasks. Furthermore, these generic models can close the perception-planning loop through common sense reasoning, applicable to both language-based tasks and open-vocabulary visual recognition. However, due to the scarcity of robotics data for these vision-language-action-based approaches and the lack of clear evaluation protocols for these models, including safety guarantees, it is hard to evaluate the real benefits of these models. In this work, we analyse three different image-based navigation models: ViNT, NoMaD, and GNM. We propose specific metrics to quantitatively evaluate the visual-action-based navigation methods in indoor, outdoor, and simulation environments. In each scenario, these methods are adapted to a different robotic setup than the one provided in the original papers, providing an opportunity to benchmark the generalizability of these methods. The test setups and the associated codebase are available on the project webpage 1.