Rubric Judgments Lack Uncertainty: Conformal Weighting for LLM Evaluation
Rubric-based evaluation is widely used to assess large language model (LLM) outputs by decomposing quality into multiple criteria and assigning each a binary judgment from an LLM verifier. However, judgment certainty varies substantially across criteria. Existing methods uniformly aggregate these judgments into a final...
Li-Cheng Pan, Hao Yang, Hao-Xuan Li et al.
· 0 citations