Label-Free Visual Concept Drift Detection via Classifier Two-Sample Tests and Characteristic Function Embeddings
TL;DR
This work proposes a real-time, label-free concept drift detection framework that couples the Classifier Two-Sample Test (C2ST) with a novel Characteristic Function Embedding (CFE) bottleneck, operating directly on deep convolutional representations.
Abstract
Deploying deep neural networks in non-stationary environments exposes them to concept drift, silently degrading predictive performance over time. Traditional statistical two-sample tests and distance-based heuristics frequently fail to detect natural, semantic shifts in high-dimensional visual streams. To address this critical blind spot, we propose a real-time, label-free concept drift detection framework that couples the Classifier Two-Sample Test (C2ST) with a novel Characteristic Function Embedding (CFE) bottleneck. Operating directly on deep convolutional representations, this architecture compresses features to resolve dimensionality constraints and ensure highly efficient downstream evaluation. We systematically benchmark our sliding-window framework against classical and modern baselines, including Maximum Mean Discrepancy (MMD), Kolmogorov-Smirnov (KS), and DriftLens, under complex, real-world shifts using the WILDS Waterbirds subpopulation dataset and the PACS domain generalisation benchmark. Our empirical findings demonstrate that distance-based baselines severely degrade under semantic shift; notably, the KS test collapses to a 0.45 score on the Waterbirds dataset, while MMD shrinks by an order of magnitude. In contrast, the proposed C2ST framework retains highly robust discriminative power, achieving above 0.97 AUROC on semantic shifts and saturating effectively on domain shifts. To complement this, a micro-sensitivity analysis on the CIFAR-10 dataset demonstrates that our framework successfully isolates ultra-low intensity synthetic Gaussian drift, maintaining detection capabilities where traditional tests fail. Ultimately, the framework maintains a calibrated false-positive rate on stable, in-distribution streams. These results confirm that the C2ST-CFE architecture provides a mathematically grounded and highly responsive mechanism for safe machine learning deployment in real-world visual applications.