Aug 2026· IEEE Transactions on Pattern Analysis and Machine Intelligence· Vol PP· 0 citations
Medicine
TL;DR
This work proposes PointPDF V2, a unified framework that integrates open-set recognition (OSS) and incremental learning (IL) into a cohesive pipeline and introduces a more challenging continual OWSS protocol in 3D, where models must simultaneously preserve the known-class performance, acquire new knowledge, and still identify the remaining unknowns across sequential updates.
Abstract
3D semantic segmentation has achieved remarkable advances with powerful architectures and large-scale datasets. However, most existing approaches perform under closed-set assumptions, limiting their applicability in dynamic real-world environments where novel categories frequently emerge. This motivates the task of open-world semantic segmentation (OWSS), in which a model must not only identify unknown objects at inference time but also incorporate them with previously learned classes once annotated. To address this challenge, we propose PointPDF V2, a unified framework that integrates open-set recognition (OSS) and incremental learning (IL) into a cohesive pipeline. Our design consists of three components: a multistage pseudo-labeling (MPL) module that generates pseudo-labels for unknown classes by leveraging confidence and geometric information; a distribution-aware confidence estimation (DCE) module that models class-wise semantic distributions to separate known and unknown categories; and a weighted knowledge distillation (WKD) module that balances old-class retention with novel-class adaptation in IL. In addition to the standard OWSS setting, we introduce a more challenging continual OWSS (COWSS) protocol in 3D, where models must simultaneously preserve the known-class performance, acquire new knowledge, and still identify the remaining unknowns across sequential updates. Extensive experiments on both indoor and outdoor benchmarks show that PointPDF V2 consistently outperforms state-of-the-art baselines in OSS, IL, and C-OWSS. We shall release our code and models upon publication of this work.
A lightweight boundary-aware learning framework that explicitly models boundary regions during training is proposed, showing that incorporating boundary-aware supervision provides an effective and efficient approach to improving segmentation quality in challenging regions.
Waseem Iqbal, J. Paffenholz· The International Archives o...· 0 citations
Confidence-gated relational distillation is proposed, an exemplar-free teacher–student framework that combines feature-level relation preservation with semantic-level background correction that provides an effective balance between old-class retention and novel-class acquisition without introducing replay data or separate architectural branches during incremental optimization.
Lei Wang, Rongxiang Liu· Applied Sciences· 0 citations
DeepChoice is introduced, a lightweight view-weighting module for image-guided 3D semantic segmentation that is sensor-agnostic, requires no meshing, and can be integrated as a replacement for standard multi-view fusion rules.
A. Carreaud, Digre Frinde, Shanci Li et al.· The International Archives o...· 0 citations
Modern vision systems must operate in"open-world"settings, where models must recognize known categories and detect unseen or anomalous content. Conventional semantic segmentation models operate under a"closed-world"assumption, often producing overconfident misclassifications on novel content. We address open-world semantic segmentation, the joint task of segmenting known classes while detecting and grouping novel or anomalous content without additional supervision, by extending a dual-decoder baseline with a third, complementary decoder within a unified encoder-decoder design. The first decoder performs closed-set segmentation using Gaussian prototypes for known categories. The second uses contrastive feature learning to isolate unknown regions in embedding space. The third, our key contribution, is a sensitivity decoder that captures fine-grained texture irregularities and activation instabilities indicative of semantic uncertainty, which neither semantic prototypes nor contrastive norms can reliably detect. The three decoders provide genuinely complementary signals: class-level OOD distance in logit space, global feature energy in embedding space, and local activation instability across encoder scales. Experiments on Cityscapes and BDD-Anomaly show that our method improves anomaly segmentation and novel-class discovery while maintaining competitive closed-set accuracy, with gains of +2.4% AUROC and a 2.5 pp. reduction in FPR@95TPR on BDD-Anomaly over the baseline.
Anastasios Romanos Varvarigos, Nikos Giakoumoglou, Tania Stathaki· 0 citations
This work proposes a framework for detailed object-level reasoning across multiple objects with three components: MO3D, an instruction dataset requiring fine-grained multi-object comparison; Multi-3DLLM, using a minimal Patch-Interaction Transformer (PIT) that models inter-/intra-object relationships while preserving local geometry.
K. Ide, Ryousuke Yamada, Yue Qiu et al.· 1 citation· ⚡1
Recently, open-vocabulary 3D object detection (3D-OVD) has gained increasing attention for its ability to detect unseen objects in 3D scenes. Existing approaches typically adopt a two-stage pipeline that first discovers novel objects using foundation models and then trains a 3D-OVD model based on these discovered objects. Although effective, this pipeline often suffers from inaccurate localization and mismatched classification during the discovery stage, which subsequently limits the performance of the model training stage. To address these limitations, we advocate for improving both the reliability of novel object discovery and the robustness of model training, and propose an innovative framework. Specifically, for reliable discovery, our co-distillation strategy distills high-quality novel objects by applying Hungarian matching over a comprehensive score that incorporates geometric consistency, structural objectness, and semantic certainty. To enhance robust model training, we further propose a dual-guidance learning scheme, incorporating a scene-awareness-guided uncertainty regularization for the regression head and an LLM-guided hierarchical alignment for the classification head, effectively mitigating the negative effects of imprecise 3D bounding boxes and semantic ambiguity. Extensive experiments on SUN RGB-D and ScanNetV2 demonstrate that our method achieves significant performance gains over state-of-the-art approaches. Code is available at https://github.com/shangboyuan/Co-3DGT
Shangbo Yuan, Jie Xu, Xiaofeng Zhu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
