Entropy-Guided LLM Decoding via Probabilistic Circuits
This work argues that future entropy is also essential for LLM decoding, and estimates its lower and upper bounds by computing the lower and upper bounds on a hidden Markov model (HMM), which is distilled as a surrogate of the LLM.
Zhizhen Chen, Daniel Mingyi Israel, Guy Van den Broeck et al.
· 0 citations