ContrastSFT: Contrastive Logit Regularization Supervised Fine-Tuning for Mitigating Hallucinations in Large Language Models

Abstract

Large Language Models (LLMs) have achieved remarkable success in natural language generation but remain prone to hallucinations—generating content that is fluent but factually incorrect. While recent inference-time interventions like Contrastive Decoding (CD) effectively mitigate this by penalizing tokens favored by a "weak" hallucination-prone model, they introduce significant computational overhead (doubling inference latency) and fail to permanently align the model. In this paper, we propose \textbf{ContrastSFT}, a novel training framework to mitigate hallucinations in LLMs that internalizes the efficacy of contrastive decoding into the model's parameters via Contrastive Logit Regularization (CLR). Unlike standard Supervised Fine-Tuning (SFT) which indiscriminately maximizes the likelihood of ground-truth tokens, ContrastSFT dynamically recalibrates the training objective by subtracting the log-probabilities of a weak reference model. This effectively penalizes "easy" but potentially hallucinatory patterns captured by the weak model, forcing the model to learn more robust, factual representations. Extensive experiments on NLU benchmarks (ParaRel, WiCE) and Factuality tasks (HaluEval, MMLU) demonstrate that ContrastSFT achieves a 5-9\% absolute improvement over SFT and previous contrastive methods. Crucially, ContrastSFT eliminates the need for auxiliary models during deployment, retaining the high inference efficiency of standard LLMs. Code will be released.