Diffusion language models accelerate generation by predicting multiple tokens in parallel, but factorized denoisers treat simultaneous predictions as conditionally independent. Decoding heuristics reduce the resulting errors by controlling which tokens are committed, but leave the underlying distribution factorized. Jo...
Zi-Lei Shao, Ian Li, Heng Zhao et al.· 0 citations
By making objectives available to inference rather than only evaluation, COFFEE brings joint conditioning, completion-weighted guidance, and optimization-based constraints into pretrained neural generation, showing the potential of neural-symbolic methods in diffusion guidance.
Hua-Qiu Xu, Dong-Xin Li, Gwen Yidou-Weng et al.· 0 citations
This work argues that future entropy is also essential for LLM decoding, and estimates its lower and upper bounds by computing the lower and upper bounds on a hidden Markov model (HMM), which is distilled as a surrogate of the LLM.
Zhizhen Chen, Daniel Mingyi Israel, Guy Van den Broeck et al.· 0 citations
This paper formalizes the problem of KV eviction and proves that it is computationally hard, and shows that this probabilistic version of KV eviction coupled with decode time correction is more robust to different tasks compared to existing eviction methods and achieves competitive performance at the same compression b...
Renato Lui Geh, Alexander K. Chen, Daniel Mingyi Israel et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.