Xu Zhou

Data-Efficient Online Training for Direct Alignment in LLMs

This work proposes DOTA, a data selection framework that minimizes the cost of generating preference data, while still ensuring the quality of training, and proposes a theoretically grounded metric called Preference Perplexity (PFP) that enables it to design a low cost, gradient-based method to effectively estimate the...