Jul 2026
Trace-Based On-Policy Distillation for Masked Diffusion Language Models
Diffusion large language models (dLLMs) are a promising alternative to autoregressive generation. However, reasoning-oriented post-training for dLLMs remains challenging. Supervised fine-tuning (SFT) for dLLMs requires dense but often off-policy masked states, while reinforcement learning (RL) relies on sparse rewards...
Haolin Ren, Ziyang Huang, Chenhao Yuan et al.
· arXiv.org · 0 citations