Abhishek Divekar

PRECISE: Reducing the bias of LLM evaluations using prediction‐powered ranking estimation

Abstract Evaluating the quality of search systems traditionally requires a significant number of human relevance annotations. In recent times, several systems have explored the usage of Large Language Models (LLMs) as automated judges for this task while their inherent biases prevent direct use for metric estimation. W...