SAFT: Safety-Preserving Adaptation via Fine-Tuning Transfer for Large Language Models

Abstract

Adapting instruction-tuned large language models (LLMs) to downstream domains is increasingly common, yet fine-tuning on imperfect data can erode the safety alignment learned during post-training. Existing safety-preserving fine-tuning methods typically optimize the aligned instruction model directly, which can destabilize refusal behaviors or impose an ''alignment tax'' that limits task adaptation. We propose SAFT (Safety-preserving Adaptation via Fine-tuning Transfer), a safety-preserving adaptation framework that decouples task learning from alignment preservation by learning a safety-guided task update on the paired pretrained base model, rectifying task gradients to avoid conflicting directions with respect to a safety objective, and then transferring the update to the frozen instruction model via parameter-space grafting. Across mathematical reasoning, code generation, and medical question answering on two open-source model families (Llama3.1-8B-Instruct and Gemma3-4B-IT), SAFT improves downstream utility while maintaining low harmfulness under a unified evaluation protocol, and achieves better safety and utility trade-offs than nine baselines. Warning: This paper contains unfiltered content generated by LLMs that may be offensive to readers.