Krishna Chytanya Ayyagari

Pinned and Still Unstable: Within-Judge Verdict Variance and the Noise Floor of LLM-as-Judge Leaderboards

This work shows that LLM-as-Judge is operationalized on cloud serving infrastructure, not of any particular model family, and proposes a minimal, low-cost reporting protocol: several judge re-runs, published stability profiles and adjacency intervals, and results under at least two judges from different families.