FOCUS&RePAIR: Mitigating Text Degeneration via Token-Level Guidance for Pruned Large Language Models
A token-level analysis of this failure mode is presented by viewing decoding as a dynamical process that enters and persists in a small set of recurrent contexts and shows that persistence is controlled by the escape mass assigned to plausible alternatives within the token sampling set.
Junyoung Lee, Se-Hee Park, Shinhyoung Jang et al.
· 0 citations