Probability is Not Enough: Exploring and Counting Divergent Tokens for Reasoning Uncertainty Quantification in LLMs
DTC is introduced, a framework that estimates confidence by counting tokens at which two models strongly disagree during decoding, and finds that their count is almost negatively associated with answer accuracy, thereby serving as a simple yet effective signal for uncertainty quantification.
Fei-Yang Li, Sheng-Jing Liu, Qi Zhan et al.
· 0 citations