Don't Commit Alone: Joint Token Commitment in Diffusion Language Models

Abstract

Diffusion language models (dLLMs) commit multiple tokens per denoising step by decoding each selected position independently from a shared context. When these positions are dependent, this factorization introduces an error captured by conditional total correlation, which confidence-based selection cannot infer from marginal probabilities alone. We propose CoCommit, a marker-gated coordination pass that delays commitment. After the usual bundle selection, a learned marker identifies the commit set, and the backbone's last n layers are re-applied to coordinate the marked positions before greedy argmax writes the tokens. This approximates joint-mode decoding while reusing existing weights, requiring only one partial forward pass and no auxiliary model. On LLaDA 2.1 with LoRA adapters and greedy inference, joint commitment improves five of the seven evaluated benchmarks over the released factorized decoder. The largest gains occur on code and reasoning tasks, while the remaining tasks are near parity.