Daisuke Kawahara

Style-Debiased DPO: Updating LLM Knowledge with Factuality-Aware Synthetic Preference Data

Continued pretraining (CPT) with data augmentation such as paraphrasing can store inside a large language model (LLM) the knowledge of a small source corpus. The stored knowledge, however, is not always retrieved correctly. We study the eliciting side rather than the storing side: we use preference optimization, which...