Reasoning-Preserving Fine-Tuning of Post-RL LLMs with Null-Basis LoRA
Reinforcement learning (RL)-based post-training has become an effective approach for eliciting reasoning capabilities in large language models (LLMs). However, adapting post-RL models to new knowledge domains or behaviors through subsequent supervised fine-tuning (SFT) can severely overwrite these capabilities. Existin...
Wen-Zhi Fang, Nicholas Tzou, L. Valkov et al.
· 0 citations