Aug 2026· IEEE Transactions on Pattern Analysis and Machine Intelligence· Vol PP· 0 citations
Medicine
TL;DR
GSP++ is presented, a graph-based inference framework that exploits the manifold structure of test-time point clouds to refine VLM scores without additional training, and introduces a self-training strategy that selects high-confidence positive and negative samples and assigns them calibrated pseudo scores to further stabilize propagation.
Abstract
3D point cloud understanding, e.g. out-of-distribution detection and recognition, remains a challenge, particularly in applications where safe and robust perception is critical. While recent 3D VLMs enable direct recognition via text prompts, their predictions are often unstable when the test distribution shifts or when only a handful of labeled samples are available. In this paper, we present GSP++, a graph-based inference framework that exploits the manifold structure of test-time point clouds to refine VLM scores without additional training. GSP++ constructs a graph over class prototypes, optional few-shot samples, and unlabeled test instances, and performs log-scaled score propagation to improve robustness across different dataset sizes while keeping computation efficient. The framework enables unified 3D OOD detection and classification. Moreover, we introduce a self-training strategy that selects high-confidence positive and negative samples and assigns them calibrated pseudo scores to further stabilize propagation. To enable practical deployment, we introduce dynamic graph construction and maintenance strategies that efficiently manage computation and memory, enabling scalable 3D understanding under both transductive and incremental inference. Extensive experiments on six datasets show that GSP++ not only reproduces and strengthens the full suite of OOD detection evaluations from GSP, a prior conference version, but also consistently improves zero-shot and few-shot 3D classification over strong VLM baselines. The code is available on https://github.com/handsome999KK/GSP.
TextFM is presented, the first language-guided feature matching framework that incorporates domain-invariant semantic information from vision-language models (VLMs) to establish a strong foundation for robust and generalizable feature matching under real-world constraints.
Zhihao Zheng, Jinglun Feng, Nirav Savaliya et al.· 2 citations
A SAM-guided framework for point cloud oversegmentation that significantly improves boundary recall and maintains high oracle accuracy while maintaining high oracle accuracy, and generalizes well to unseen datasets without retraining, showing strong cross-dataset inference capability.
Dening Lu, Michael A. Chapman, Jonathan Li· The International Archives o...· 0 citations
3D point cloud-language models (3D-LLMs) enable 3D understanding by pairing point cloud encoders with large language models, but existing methods rely on costly multi-modal encoders (e.g., ULIP-2) that require image-text-point cloud alignment on 8x A100-scale compute, creating high barriers for research and deployment. In this work, we systematically investigate whether low-cost self-supervised point cloud encoders, specifically PCP-MAE and Point-MAE, can serve as effective alternatives. Using MiniGPT-3D as our testbed, we evaluate 7 encoder initialization/pre-training setups (1 multi-modal baseline, 5 self-supervised, 1 random init) under frozen and unfrozen fine-tuning (12 total groups), across 2 architectures (MaskTransformer, PointTransformer), 3 objectives (PCP-MAE, Point-MAE, random init), and 2 datasets (Objaverse 660K, ShapeNet55-34 approximately 50K). Our experiments reveal three key findings: (1) The four-stage MiniGPT-3D pipeline can effectively train a 3D encoder from random initialization: an end-to-end trained random init encoder reaches 52.50% open-vocabulary accuracy and 44.45 captioning score, approaching top pre-trained variants; (2) Architecture and pre-training objective show strong crossover interaction: PCP-MAE + MaskTransformer achieves 59.00% accuracy (best self-supervised), while Point-MAE + MaskTransformer drops to 46.50%, with the pattern reversed for PointTransformer; (3) Closed-set ModelNet40 classification remains a core weakness of purely geometric encoders, reaching only ~13-18% accuracy vs. ~62% for the multi-modal baseline, even after end-to-end fine-tuning. Our results offer practical guidelines for cost-effective 3D-LLM design and reveal interaction patterns between self-supervised objectives and encoder architectures.
Generalized few-shot 3D point-cloud segmentation (GFS-PCS) asks a model to segment a scene into many base classes seen at training time and a set of novel classes. The state of the art reaches novel classes by reconciling a dense but noisy 3D vision-language prior with the few-shot support, but it pays for this with base 3D labels, per-episode training, and the support annotations themselves. We ask how far the same reconciliation can go with none of these: no training, no 3D labels, and not even the few-shot support. We pair a frozen 3D vision-language model (RegionPLC) as a dense prior with a frozen promptable concept segmenter (SAM3), prompted by the bare novel class names and lifted from posed RGB views, and reconcile the two by cross-view consistency: a point becomes novel only when enough of the views that see it agree. On the ScanNet200 GFS-PCS benchmark this fully training-free, open-vocabulary pipeline improves novel mIoU by +2.6 over the training-free dense prior while holding base accuracy within 0.5, and recovers a third (33%) of the novel-class gap to the trained state of the art that uses far more supervision. We further show that injecting the few-shot support into the pipeline, as a fusion gate and as a prototypical dense classifier, adds nothing over consistency alone and in fact degrades it through the classifier, which is why the method needs no support at all. On the harder ScanNet++ benchmark, where the dense prior is far weaker on novel classes, the same pipeline nearly doubles novel mIoU (+15.7, from 16.2 to 31.9) at a 1.7 base cost, lifting the harmonic mean from 21.5 to 31.1
Silas kwabla Gah, E. Owusu· arXiv.org· 0 citations
Qwen-3D is introduced, a geometry-aware LMM that compresses visual information within the Qwen backbone using multi-view geometric cues, enabling efficient long-horizon visual reasoning over static scenes and incorporates a query-based segmentation decoder that grounds language directly in the underlying 3D scene representation.
Lucy Lin, Ayush Jain, Yifan Liu et al.· 0 citations
DeepChoice is introduced, a lightweight view-weighting module for image-guided 3D semantic segmentation that is sensor-agnostic, requires no meshing, and can be integrated as a replacement for standard multi-view fusion rules.
A. Carreaud, Digre Frinde, Shanci Li et al.· The International Archives o...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
