When LLMs Benchmark Themselves: Deconstructing Self-Bias in Automated Evaluation
This work finds that self bias arises from two compounding sources, LLM as a testset and LLM as an evaluator, and their combination amplifies the effect and confirms that the phenomenon extends to open-ended generation on the Chatbot Arena task.
Wenda Xu, Sweta Agrawal, Vilém Zouhar et al.
· 2 citations