Cheng-Ting Yu

HiLo-RD: Cross-Budget Self-Distillation for Masked Diffusion Language Models

We propose HiLo-RD (High-to-Low Budget Representation Distillation) to improve masked diffusion language models (MDLMs) under a limited generation budget. Unlike autoregressive models, MDLMs choose both what to generate and where to commit it. This creates two coupled challenges: predicting useful content before reason...