Understanding and Exploiting Anisotropy in Post-Training
LLM post-training combines supervised fine-tuning (SFT), a mode-covering forward-KL objective, with reinforcement learning (RL), a mode-seeking reverse-KL objective, with reinforcement learning (RL), a mode-seeking reverse-KL objective to exploit anisotropy, a division of labor that post-training can exploit.