Intrinsic dimension for evaluating and analyzing neural networks for medical image segmentation.

Background

AND

Objective

Evaluating medical image segmentation models typically relies on expert annotations, which are scarce, costly, and time-consuming to obtain. Intrinsic dimension (ID), defined as the minimal number of parameters required to describe high-dimensional data, has been shown to correlate with model generalization in classification tasks, but whether this relationship extends to segmentation remains unclear.

Methods

In this study, we systematically investigate ID as a label-free indicator for evaluating segmentation performance, including the generalization of standard segmentation models and the adaptability of promptable foundation models (i.e., SAM-based models). We also perform a layer-wise ID analysis to characterize variations in latent representation complexity. Furthermore, we propose a novel, computation- and memory-efficient strategy for ID calculation to broaden its applications.

Results

We reveal the following findings: 1) For standard segmentation, lower ID values of training-sample representations are consistently associated with better generalization performance on unseen test samples, as measured by DSC. 2) For promptable segmentation, test-data ID consistently reflects model adaptability to specific target data compared to entropy, a metric commonly believed to predict performance. 3) Layer-wise ID in segmentation follows an initially increasing and then decreasing pattern, similar to that in classification.

Conclusions

Our findings suggest the potential of ID as a label-free indicator for assessing the generalization and adaptability of segmentation models, thereby supporting model selection in label-scarce clinical settings.