LoRetta is proposed, a foundation model coupling matchability-aware affine localization with guided dense registration with LEVIR-GM, a global-scale multi-temporal optical matching benchmark with dataset-native matchability labels and establishes a unified evaluation protocol for sparse, semi-dense, and dense matchers.
Abstract
Dense image matching establishes pixel-wise correspondences and underpins broad applications in computer vision and photogrammetry. However, extending dense matching to global-scale remote sensing remains challenging because image pairs may differ in acquisition time, season, viewpoint, spatial resolution, and land-cover state. The resulting large geometric offsets, partial overlap, and intrinsically unmatchable regions make direct dense correspondence prediction unreliable and inefficient. We thus reformulate dense matching as localization-and-registration: first localizing the matchable overlap and affine geometry, then refining dense residuals within the aligned frame. Based on this formulation, we propose LoRetta, a foundation model coupling matchability-aware affine localization with guided dense registration. We also introduce LEVIR-GM, a global-scale multi-temporal optical matching benchmark with dataset-native matchability labels (103K aligned, 827K augmented pairs, six continents, five years, 0.5-1024 m resolution). We further establish a unified evaluation protocol for sparse, semi-dense, and dense matchers. On LEVIR-GM, LoRetta achieves an area under the curve (AUC) of 83.3%, outperforming the strongest baseline RoMa v2 by 1.6 points, with larger percentage of correct keypoints (PCK) gains of 6.5 and 8.2 points at 1 and 2 pixels, while reducing inference latency by 47.8%. Astronaut-to-satellite and unmanned aerial vehicle (UAV)-to-satellite geolocalization experiments further demonstrate its transferability as a reusable geometric aligner.
Co-registration underlies nearly every multi-temporal and multi-sensor use of optical satellite imagery, and operational products still carry documented offsets well above the fraction-of-a-pixel scale at which change detection, time series, and data fusion degrade. Real image pairs differ along several axes at once (sensor response, scene content, viewing geometry, resolution, mosaic seams), and the last of these is not a single global motion. Existing tools embed a motion model and constants tuned to their development data; a pair that fits is registered precisely, while one that does not either fails to match or returns a result wrong by tens of pixels with no failure reported. Learned matchers add a GPU requirement and carry no accuracy guarantee outside their training distribution. We present SCDF (self-calibrating displacement fields), a training-free, GPU-free estimator whose motion model is the dense per-pixel displacement field itself, so no scene motion falls outside the model. A single predict--measure--filter loop runs over a resolution pyramid: the accumulated field predicts where each patch of the moving image falls in the reference, RootSIFT matching and a correlation pass measure the displacement there to sub-pixel precision, and filters whose thresholds are all calibrated on the image pair itself decide what survives. One configuration, with no per-dataset tuning, processes full $8192^2$ scenes on a single CPU core. On 584 constructed-ground-truth pairs built from real Sentinel-2, Landsat-8/9, and NAIP imagery, against seven classical baselines and two zero-shot pretrained matchers, SCDF registers every pair with zero failures, reduces the best baseline's real-pair median end-point error from 6.83 to 4.17m, and cuts its 90th percentile from 17.8 to 7.77m.
Shoukun Sun, Zhe Wang, S. Salati et al.· 0 citations
Remote sensing image matching is a fundamental task for numerous Earth observation applications, for example, change detection, multitemporal analysis, and cross-sensor fusion. However, seeking reliable matching remains highly challenging due to substantial variations in imaging conditions. Existing approaches often suffer from limited robustness when confronted with large geometric deformations. To address this, we propose a novel optimal transport (OT)-guided semantic consensus framework. Unlike traditional consensus methods that rely on the spatial topology of feature points, our approach leverages visual semantic cues to enhance the discriminability of putative matching. Specifically, by integrating visual–language large models, we design a semantic consensus learning paradigm, which can be used to evaluate the visual semantic consistency of correspondences. More importantly, grounded in OT theory, our semantic consensus learning framework can be trained in a self-supervised manner without requiring additional manual annotations. In this work, we also provide a model instantiation that integrates the proposed semantic consensus learning with mismatch rejection networks. Overall, this work provides a general paradigm for integrating self-supervised learning with local semantic consensus learning. Comprehensive evaluations on multiple datasets show that our method not only outperforms state-of-the-art baselines in matching accuracy but also generalizes well to diverse image deformations.
Manman Peng, Yuwen Huang, Wenting Han et al.· IEEE Journal of Selected Top...· 0 citations
Ground image localization with respect to satellite imagery is a key enabler for metrically-accurate, geo-localized 3D scene reconstruction from unconstrained image collections. Existing cross-view localization methods have strict requirements such as panoramic imagery or known initial locations, limiting their applicability for in-the-wild reconstruction settings. We propose a robust hierarchical cross-view localization framework that leverages geometric constraints from Structure-from-Motion (SfM) models derived from unconstrained ground image collections. Our method generates coarse-to-fine pose hypotheses through a cross-view matching approach and aggregates noisy predictions across SfM model(s) using Kernel Density Estimation to recover consensus alignments while filtering outliers. Experiments demonstrate reliable localization performance from challenging image collections. Empirically we found satellite-referenced alignment enables accurate metric scale estimation, doppelg\"anger detection, and merging of disjoint SfM reconstructions, resulting in more complete, geo-localized site models than are possible with SfM alone.
A. Daruna, Ben Southall, Niluthpol Chowdhury Mithun et al.· 0 citations
Side-scan sonar (SSS) mosaicking plays a crucial role in large-scale seabed mapping but is challenged by complex nonlinear, spatially varying distortions due to diverse sonar acquisition conditions. Existing rigid or affine registration methods fail to model such complex deformations, whereas traditional nonrigid techniques tend to overfit and lack robustness in sparse-texture sonar data. To address these challenges, we propose a coarse-to-fine hierarchical nonrigid registration framework tailored for large-scale SSS images. Our method begins with a global thin plate spline (TPS) initialization from sparse correspondences, followed by superpixel-guided segmentation that partitions the image into structurally consistent patches preserving terrain integrity. Each patch is then refined by a pretrained SynthMorph network in an unsupervised manner, enabling dense and flexible alignment without task-specific training. Finally, a fusion strategy integrates both global and local deformations into a smooth, unified deformation field. Extensive quantitative and visual evaluations demonstrate that our approach significantly outperforms state-of-the-art rigid, classical nonrigid, and learning-based methods in accuracy, structural consistency, and deformation smoothness on the challenging sonar dataset.
Can Lei, Nuno Gracias, Rafael Garcia et al.· IEEE Transactions on Geoscie...· 0 citations
Dense image matching is crucial in applications such as 3D reconstruction, autonomous driving, and remote sensing mapping; however, weak textures, occlusions, and large-disparity scenes remain challenging. To address these issues, this paper proposes a dense matching network based on a Transformer and multi-scale feature fusion, called Task-aware Multi-Scale Matching Network (TMSMNet). First, Swin Transformer is used to model global context in feature maps, enhancing the feature discriminability in weak texture regions. Then, a multi-scale cost volume is constructed, and adaptive fusion is achieved through deformable convolution to accommodate disparity variations of different ranges. Finally, an attention- guided iterative optimization module is introduced to improve the matching accuracy in occluded regions. Experimental results on the Scene Flow, KITTI-2015, and Middlebury datasets show that TMSMNet outperforms mainstream methods such as RAFT-Stereo on the D1-all metric of KITTI- 2015 and demonstrates good generalization and robustness. Ablation studies also confirm the effectiveness of each module. In summary, the method in this paper provides a feasible approach for dense matching. Future work will explore model lightweighting to support real-time applications and attempt to combine generative models to handle completely textureless regions, further enhancing its performance in complex scenes.
Abstract. Superpoint matching is a critical step in coarse-to-fine point cloud registration, and its performance directly affects the accuracy of subsequent point matching and pose estimation. However, most existing methods establish correspondences mainly relying on feature similarity, without explicit modeling of spatial structure, which easily leads to unstable matching in complex scenarios such as noise, occlusion, and low overlap. To address these issues, this paper proposes a coarse-to-fine point cloud registration method guided by prior correspondences. First, prior superpoint correspondences are constructed using rigid transformations estimated by existing SOTA methods, and are serially encoded via a prior encoding module to provide explicit constraints for feature learning. Furthermore, multiple geometric information including pairwise distances, angles, and normals is introduced and uniformly encoded to enhance spatial struc-ture representation. On this basis, a prior-guided sparse mixture-of-experts attention mechanism is designed to differentially model features in overlapping and non-overlapping regions, thereby improving feature discriminability and structural consistency. Using the learned features, the model gradually establishes correspondences through superpoint matching and point matching, and estimates the final rigid transformation with RANSAC. Experiments on the 3DMatch dataset show that when sampling 1000 point correspondences, the proposed method achieves an inlier ratio of 80.7% and a registration recall of 92.9%, which are 5.5% and 1.1% higher than the baseline method respectively, verifying the effectiveness of the proposed method in terms of accuracy and robustness.
Meng Sun, Juntao Yang, Yutao Zhang et al.· The International Archives o...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
