This work introduces SPEAR (Self-Play Enhancement via Advantage-Weighted Refinement), an efficient online learning algorithm for federated LLM fine-tuning that requires only an interaction-level binary satisfaction signal together with non-answer feedback, without requiring a ground-truth answer to be provided, and doe...
Seohyun Lee, Wen-Zhi Fang, Dong-Jun Han et al.· arXiv.org· 1 citation
Fire (Fisher-Informed REcalibration), a dual-branch framework that recalibrates the supervision applied to correct and incorrect on-policy outputs during fine-tuning, is proposed, which provides substantially more stable self-distillation while maintaining strong downstream performance, particularly in settings where s...
Seohyun Lee, Dong-Jun Han, Seyyedali Hosseinalipour et al.· 0 citations
This work proposes a two-stage generator-in-the-loop alignment framework that consistently outperforms rank-order, random, and REPLUG-style likelihood baselines under various alignment losses and pool size settings, suggesting that answer-level generator feedback is an effective supervision signal for preference alignm...
Zhang-Yu Chang, Dong-Jun Han, Seyyedali Hosseinalipour et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.