No Universal Signal Predicts Sample-Level LLM Regression under Version Updates
It is found that signal effectiveness is task-dependent: confidence is strongest on MCQ and simpler math, while likelihood/KL signals give the most frequent gains on harder math and code; no signal is universally best across model updates either, and some cross-version signals stay informative even when confidence fails, including without labels.