A reference-based framework for examining coherence and diversity through three perspectives is presented: aligning their evolution with human trajectories, comparing their summaries with a human continuation of the same prompt, and estimating their likelihood under a human reference distribution.
Probability-floor samplers substantially narrow the blind spot, demonstrating that the choice of truncation criterion matters for retaining human-used tokens and motivate decoding methods that preserve contextually appropriate low-probability choices while maintaining generation quality.
E. Arias, Nurzhan Sapargali, Christian Heumann et al.· arXiv.org· 2 citations
In experiments across open-ended generation, factual QA, and mathematical reasoning, it is shown that VCM consistently mitigates the likelihood trap, and integrates with existing decoding strategies, improving diversity and coherence and, particularly at higher decoding temperatures, reasoning accuracy.
Yuanhao Ding, Meimingwei Li, E. Arias et al.· arXiv.org· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.