Semantic-guided 3D Gaussian splatting for sparse-view reconstruction in industrial digital twins

Abstract

Novel view synthesis, which is essential for three-dimensional (3D) reconstruction, traditionally requires dense image sets and pre-calibrated camera parameters. However, in industrial digital twin applications spatial constraints often limit data acquisition to sparse views that fail to provide the feature correspondences necessary for accurate modeling. To overcome this bottleneck, a semantic-guided 3D Gaussian splatting (3DGS) framework tailored to sparse-view industrial reconstruction was introduced. Rather than relying on precise initialization and dense inputs, the proposed method couples explicit 3D Gaussian representations with the dense and unconstrained stereo 3D reconstruction (DUSt3R) end-to-end pose estimation model. The severe occlusions, typical of factory environments, are handled by incorporating Segment Anything Model 2 (SAM2) to hierarchically decompose the scene, yielding a structured representation that improves optimization stability. A probability density field-driven algorithm is subsequently applied to extract lightweight meshes directly from the optimized Gaussian point clouds. Evaluations on the MipNeRF360 benchmark and a custom industrial dataset demonstrated substantial improvements in the structural similarity index measure. By enabling robust reconstruction from limited viewpoints, this pipeline offers a practical geometric foundation for automated inspection and remote equipment monitoring.