Hunk-Constrained DPO: Segment-Level Optimization for Secure and Correct LLM Code Generation
TL;DR
Hunk-Constrained Direct Preference Optimization is introduced, a training framework that unifies security hardening and functional correction in large language models and demonstrates that HPO achieves substantial security improvements—up to 28 percentage points—while preserving or enhancing functional correctness.
Abstract
Large Language Models (LLMs) have been widely applied in code generation tasks like code completion and automated development, demonstrating significant potential for improving coding efficiency. However, research has shown that LLM-generated code frequently contains security vulnerabilities, raising concerns about its reliability in production environments. To address these security issues, various mitigation approaches have been proposed, but these methods typically impact the LLM’s ability to generate functionally correct code, which may limit their practical application in real-world development environments. In this work, we address this problem through a key observation: security patches and functional bug fixes in real-world software exhibit structural similarities as small, localized modifications. This shared characteristic suggests that a unified learning model could address both objectives jointly. Building on this insight, we introduce HPO (Hunk-Constrained Direct Preference Optimization), a training framework that unifies security hardening and functional correction. Our framework features two key technical components: a novel segment-weighted preference optimization objective to focus learning on repair logic, and an automated data synthesis pipeline to provide high-quality training data. Experiments across multiple models and programming languages demonstrate that HPO achieves substantial security improvements—up to 28 percentage points—while preserving or enhancing functional correctness.
