Skip to content
Open access

LLM-Enhanced Semantic Segmentation of Large-Scale Urban LiDAR Point Clouds via Contextual Prompting

Jul 2026 · The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences · 0 citations · 8 references

TL;DR

A contextual cue framework of "LiDAR semantics-large language model-retrieval-enhanced generation" is proposed, providing a new technical path for high-precision, fine-grained semantic segmentation of urban LiDAR point clouds and also offers a theoretical reference for promoting the in-depth application of large language models in 3D spatial intelligence.

Abstract

Abstract. As a key carrier of 3D spatial information, the semantic segmentation of urban LiDAR point clouds directly impacts the reliability of applications such as autonomous driving and digital twins. However, existing methods face two core bottlenecks: firstly, insufficient adaptation to scene-specific semantics, and secondly, an inference gap between LiDAR structured semantics and segmentation instructions, which makes it difficult to effectively combine the reasoning ability of large language models with LiDAR geometric semantics. To address these issues, this paper proposes a contextual cue framework of "LiDAR semantics-large language model-retrieval-enhanced generation". The framework first designs a lightweight semantic mapping module to convert the structured information inherent to LiDAR into natural language cues that can be understood by LLMs. Secondly, it constructs a LiDAR semantics-text vector library, utilizing the RAG mechanism to retrieve fine-grained knowledge of similar scenes in real-time, generating precise segmentation cues that include geometric features and contextual relationships. Finally, through a three-stage progressive training strategy, it guides LLMs to gradually learn the mapping relationship from semantic understanding to segmentation instruction generation.Ablation experiments verify the effectiveness of each module, and the inference efficiency meets the real-time processing requirements of large-scale urban data. This study provides a new technical path for high-precision, fine-grained semantic segmentation of urban LiDAR point clouds and also offers a theoretical reference for promoting the in-depth application of large language models in 3D spatial intelligence.

Read PDF

Similar papers

Review Open access Sep 2026

GFE-Net: Geometry-Enhanced Feature Extraction Network for Semantic Segmentation of Large-Scale LiDAR Point Clouds

Accurate semantic segmentation of large-scale outdoor LiDAR point clouds remains a challenging endeavor, primarily due to ambiguous class transitions at object interfaces, non-uniform sampling density across the surveyed area, and shared geometric signatures among distinct object categories. This paper proposes GFE-Net (Geometry-Enhanced Feature Extraction Network), a hierarchical encoder–decoder architecture that systematically improves per-point feature characterization through three complementary design contributions: First, to mitigate the shortcomings of conventional fixed-neighborhood queries in regions of variable point density, a Structure-Guided Neighborhood Adaptation (SGNA) module is devised. At its core lies a morphology-driven contextual gating (MCG) unit that synthesizes neighbor-wise calibration weights from hierarchical shape descriptors fused with elevation difference statistics, allowing the network to preferentially amplify morphologically congruent neighbors while dampening spurious or cross-boundary contributions. Second, to strengthen semantic discrimination beyond what spatial locality alone affords, a Local–Global Interactive Enhancement (LGIE) module is presented. The LGIE module simultaneously distills precise local structure through Euclidean-space neighborhood graphs and captures scene-wide co-activation patterns via compact bilinear factorization of the latent feature space, merging both streams through a residual refinement mechanism that markedly improves inter-class separability. Third, to enforce label consistency at object interfaces without relying on post-processing heuristics, a Neighborhood Prediction Consistency (NPC) loss is introduced. Built upon a Gaussian distance-decay weighting kernel, the NPC loss assigns progressively stronger penalties to label mismatches between a query point and its geometrically proximate neighbors, thereby promoting spatially coherent predictions and attenuating boundary noise. GFE-Net is rigorously benchmarked on two widely adopted large-scale datasets—S3DIS and SensatUrban—yielding OA/mIoU of 89.6%/73.1% and 93.3%/61.1%, respectively. These results demonstrate competitive performance under the reported protocols. Detailed ablation studies and computational profiling further substantiate the efficacy of each individual component.

Unknown authors · 0 citations
Aug 2026

LERPNet: a lightweight and efficient range view point cloud semantic segmentation network

This paper replaces the Stem layer in FRNet with the proposed FD-Stem, which improves feature representation while reducing computational complexity, and introduces long-range modeling capability with limited additional parameters, enabling effective learning of both spatial and channel-wise representations.

Ya-Dong Guo, Jing Liu, Wei Zheng et al. · 0 citations
Open access Aug 2026

Text-Guided Semantic Segmentation Method for Indoor 3D Point Clouds

Results indicate that incorporating textual semantic priors can effectively enhance high-level semantic representations of point clouds, providing a feasible solution for indoor 3D scene understand.

Jinyu Tan, Juntao Yang, Yutao Zhang et al. · 0 citations
Jul 2026

Range-FDSeg: LiDAR semantic segmentation based on fusion interactive learning and dynamic sampling for autonomous driving scenarios.

LiDAR semantic segmentation is significant in applications such as autonomous driving and robot navigation, as it greatly improves scene perception and object detection. However, the existing methods face the challenges of achieving high segmentation accuracy while maintaining low computational cost and complexity. In this paper, we propose a new, to our knowledge, efficient and accurate semantic segmentation network for LiDAR, called Range-FDSeg. To reduce the risk of information compression and loss when projecting 3D point cloud data onto 2D range images, we design a multi-channel fusion interactive learning (FIL) module. This module effectively integrates multimodal channels, such as coordinates, depth, and reflectivity, for interactive learning. As a result, FIL module can reduce the noise interference inherent in individual channels and capture the underlying relationships between different physical quantities. To further improve the performance, we introduce a lightweight and dynamic upsampler, called Dysample-S+. It effectively resolves the inherent challenges of traditional sampling methods through its adaptive weighting mechanism, which dynamically adjusts to local geometric patterns and density variations in raw point clouds. Extensive evaluations on publicly available benchmark datasets, including SemanticKITTI, SemanticPOSS, and NuScenes, demonstrate that the proposed Range-FDSeg outperforms most existing state-of-the-art methods.

Hong Tan, Fen Chen, Tingna Liu et al. · 0 citations
Open access Jul 2026

Unifying Street Scene Point Cloud Semantic Segmentation with Deformable Mesh-based Neural Representation

Abstract. Accurate semantic segmentation of urban point clouds is important for applications such as urban planning and autonomous driving. Recently, neural scene representations have been extended to merge semantic information across modalities and spatial dimensions. While 3D Gaussian Splatting (3DGS) enables efficient and high-quality reconstruction, its semantic understanding performance in street scenes is influenced by trajectory-constrained viewpoints, where Gaussian densification introduces occlusions and semantic ambiguity. This paper explores the use of NeRF-based neural representation for street scene point cloud semantic segmentation. Specifically, deformable neural mesh primitives (DNMPs) are used to compactly represent spatial geometry and simplify ray sampling. Then, neural fields including density, RGB, and semantics are constructed based on mesh vertex feature interpolation and MLPs. The sampled neural field values are accumulated via ray rendering and supervised using original images and corresponding semantic label maps generated by pre-trained models. Point cloud semantics are then predicted by interpolating neighboring samples within the learned field. The method is validated on the KITTI-360 and Waymo datasets. Results show that the proposed approach achieves improved semantic segmentation performance while maintaining competitive rendering quality, and supports both novel view synthesis and semantic rendering.

Yuzhou Zhou · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.