Jul 2026
The Test Oracle Problem in Synthetic LLM-as-Judge Corpora: Disappearance, Distortion and a Validation Protocol
A case in which an LLM-as-judge generation step silently failed, and it is argued that the failure mode is structural rather than incidental, and a validation protocol for analysts working in the oracle-less regime that is argued describes most contemporary multilingual LLM-as-judge corpora.
Serkan Balli
· arXiv.org · 0 citations