Book
Open access
Nov 2026
LLMJudge: Meta-Evaluating Automatic Relevance Judgments
Large language models (LLMs) are increasingly used to produce the relevance labels that underpin the evaluation of information retrieval (IR) and retrieval-augmented generation systems. Yet delegating relevance assessment to an LLM raises a meta-evaluation question that is hard to answer with existing resources: when,...
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi et al.
· Proceedings of the 35th ACM... · 0 citations