CM-DPO: Constraint-Margin Direct Preference Optimization for LLM Planning
Constraint-Margin DPO (CM-DPO), which replaces DPO's binary preference signal with a continuous margin derived from a deterministic symbolic verifier and scaled by violation severity, is introduced.