2026
Optimizing Large Language Models for Robust Domain-Specific Text-to-SQL: From Prompting to Preference Alignment
This work compares Proximal Policy Optimization (PPO), Direct Preference Optimization (DPO), and Odds Ratio Preference Optimization (ORPO) using a novel reward modeling approach based on execution and semantic principles, revealing that while standard PPO suffers from reward sparsity and catastrophic collapse on 7B models, monolithic alignment via ORPO scales efficiently to 20B parameter models.
Noah Hampp, Katya Mirylenka, Michael R. Glass
· Swiss Text Analytics Confere... · 1 citation