Beyond Benchmark Scores: An Analysis of Resolution, Spatial Structure, Illumination, and Seed Sensitivity in Training-Free Industrial Anomaly Localization on MVTec AD 2

Abstract

Training-free industrial anomaly localization avoids category-specific model training, but benchmark scores alone provide limited evidence about operational reliability. We present a fixed-protocol study on all eight MVTec AD 2 categories, comparing five variants across three families: three DINO configurations that jointly differ in transformer block and matching/reference construction, a PatchCore-style WideResNet baseline, and a PaDiM-inspired ResNet18 positional-Gaussian baseline. DINO contrasts are therefore interpreted at the complete-configuration level rather than as causal isolation of feature depth alone. The evaluation separates image ranking, normal-only calibration, pixel localization, normalized area under the per-region-overlap curve (AU-PRO), false-positive area, illumination consistency, score aggregation, defect-to-patch resolution, implementation fidelity, and stochastic sampling. No representation is uniformly best. On can, tiny-defect localization remains ineffective across all three families, with best Pixel Average Precision of $1.50\times 10^{-4}$ and mean intersection-over-union of $1.00\times 10^{-3}$ . DINO-B9 has the highest macro AU-PRO at both evaluated false-positive-rate limits (0.338 at 0.05 and 0.596 at 0.30), although category-bootstrap intervals overlap broadly. On sheet metal, same-position DINO is reproducibly best for Pixel Average Precision and thresholded intersection-over-union, whereas DINO-B9 is better under AU-PRO. One-factor audits over smoothing, pixel thresholds, image-score aggregation, and conformal operating levels preserve the central conclusions. Private illumination shifts change 30.9%–46.1% of calibrated decisions. These results show that reliable evaluation should separate representation quality, aggregation, calibration, region overlap, illumination robustness, implementation fidelity, and reproducibility rather than rely on a single benchmark score.