On Trajectory-Aware Training for Masked Diffusion Language Models
Masked diffusion models (MDMs) generate text by unmasking several tokens per step, but they are trained and sampled under different conditions. The model is trained on randomly masked sequences, whereas inference follows a trajectory shaped by the model's own predictions. Additionally, each step has no access to what t...
Manuel Madeira, Amitis Shidani, Alice Bizeul et al.
· 0 citations