Capturing Appearance and Material Optical Properties for Vision Based Navigation
Abstract
Vision-Based Navigation (VBN) must operate over surfaces whose appearance is dictatedby their material optical properties. Because flight-representative imagery with 3D groundtruth is scarce, learning-based VBN relies on simulation — and a simulator is only asfaithful as the reflectance model it uses. This paper follows that dependency end to endfor the Moon use-case. We first describe how physically-based lunar imagery is actuallyproduced — ray tracing against real Digital Elevation Models (DEMs) in the SurRenderengine — and why existing lunar datasets fall short. Isolating the component that governsphotometric fidelity, we review the Hapke model and the parameter maps available for theMoon: for lack of resolved maps, every operational renderer assumes spatially uniformreflectance, and the simulated imagery is photometrically too smooth. We address this witha geometry-to-reflectance deep learning framework that predicts spatially varying BRDF(SVBRDF) parameters directly from terrain geometry, trained by differentiable renderingagainst real orbital imagery, which reduces photometric error by 38 % on a geographicallyheld-out region of the Tycho crater. Consolidating both, we release a public benchmarkof over 130,000 samples with joint geometric and photometric supervision under diverseillumination, and validate it on VBN tasks: 3D foundation models (MASt3R, VGGT) fine-tuned on this data recover lunar relief where their pretrained counterparts fail, generalizeto terrain never seen during training, and transfer to real Chang’E-3 descent imagery;and feature matching against real orbital images confirms that the improved reflectancealigns nearly three times more accurately than Hapke under identical geometry. This papersynthesizes a coherent line of work [1, 2, 3], emphasizing the thread that links appearancemodeling to operational VBN.