Revisiting Visual Localization: A Feed-Forward Localization Framework With a Lightweight Scene Representation

Abstract

Visual localization is a key technology in many vision-based measurement applications, aiming to estimate the camera pose of a query image in a known environment. However, most existing methods rely on heavy scene-specific representations, such as explicit 3-D map construction or per-scene training. Constructing and maintaining such representations introduces nonnegligible computational overhead, storage burden, and long-term maintenance costs. To address this issue, we propose a novel visual localization pipeline that uses a set of posed reference images as a lightweight scene representation and localizes query images without explicit 3-D map construction or scene-specific training. Specifically, we exploit a geometric foundation model to infer local multiview geometry from the query image and its retrieved references. Since the predicted geometry is expressed in an arbitrary local coordinate system with unknown scale, a key challenge is how to recover an accurate metric pose of the query image from such local predictions. To address this challenge, we design a global pose recovery strategy that first registers the predicted local geometry to the world coordinate system through joint center–orientation similarity alignment using the posed reference images as global anchors, and then refines the query pose by optimizing query-associated 3-D landmarks under multiview 2-D–3-D geometric constraints. The experimental results on multiple benchmark datasets show that our method achieves competitive localization performance and improved robustness under sparse reference-view settings and challenging viewpoint or appearance variations, reducing the average translation and rotation errors of the strongest Unseen baseline from 55 cm and 0.56° to 13 cm and 0.23° on Cambridge Landmarks, respectively.