Reducing Pretraining-Generation Mismatch in Diffusion Language Models
Prefix-Conditioned Diffusion (PCD), which samples a boundary, preserves the prefix, preserves the prefix, and denoises the suffix, is proposed, and improvements in reasoning and coding over native diffusion training are shown.
Xiao-Cheng Lu, Hua-Bin Liu, Song Guo et al.
· 0 citations