When Models Leave the Training Distribution: A Tutorial on OOD Detection
TL;DR
This tutorial covers recent developments in OOD detection from both theoretical and practical perspectives, including four major categories: (1) post-hoc methods, (2) training-based methods with auxiliary outliers, (3) training methods without auxiliary outliers, and (4) foundation model-based approaches, along with recent advances in each area.
Abstract
Despite their remarkable success, Deep Neural Networks (DNNs) remain challenging to deploy in critical applications due to their inability to handle out-of-distribution (OOD) data. DNNs are trained assuming training and deployment data follow the same distribution, but this assumption is often violated in practice. When encountering unfamiliar inputs, models may produce highly confident yet incorrect predictions, and in high-stakes decision-making environments, such incorrect predictions can be costly. OOD detection has thus become fundamental across deep learning, affecting applications from computer vision and NLP to security, autonomous systems, and generative models. This tutorial covers recent developments in OOD detection from both theoretical and practical perspectives, including four major categories: (1) post-hoc methods, (2) training-based methods with auxiliary outliers, (3) training methods without auxiliary outliers, and (4) foundation model-based approaches, along with recent advances in each area.