Heewoo Jun

Entropy-Guided LLM Decoding via Probabilistic Circuits

This work argues that future entropy is also essential for LLM decoding, and estimates its lower and upper bounds by computing the lower and upper bounds on a hidden Markov model (HMM), which is distilled as a surrogate of the LLM.

Jailbreaking Jailbreaks: A Proactive Defense for LLMs

P RO A CT represents an orthogonal defense strategy that serves as an additional guardrail to enhance LLM safety against the most effective attacks.

SciPedia: Unlocking the Value of Scientific Data for Pre-training