Detecting Hard Examples for Semantic Segmentation with Auxiliary 3D LiDAR

Abstract

In this study, we propose a method for automatically detecting hard examples from unlabeled real-world data that lead to performance degradation in semantic segmentation models. The proposed method estimates obstacle regions using geometric information from a 3D LiDAR and generates obstacle masks by projecting them onto the image plane. These masks are compared pixel-wise with the semantic segmentation results, and a score based on the misclassification rate within obstacle regions is computed for each image. For evaluation, a semantic segmentation model trained on data collected at the Ikuta Campus of Meiji University was applied to unseen real-world data collected on the Tsukuba Challenge 2025 verification course. Experimental results confirmed that the proposed method could effectively identify images with many misclassified obstacle pixels.