Rubric-based evaluation is widely used to assess large language model (LLM) outputs by decomposing quality into multiple criteria and assigning each a binary judgment from an LLM verifier. However, judgment certainty varies substantially across criteria. Existing methods uniformly aggregate these judgments into a final...
Li-Cheng Pan, Hao Yang, Hao-Xuan Li et al.· 0 citations
Rubrics provide structured, fine-grained signals for training and evaluating large language models (LLMs). Yet reliable query-specific rubrics are difficult to construct. Existing approaches often derive supervision from human-written rubrics, preference data, or sampled responses. Direct query-to-rubric generation avo...
Hao Yang, Licheng Pan, Xiaoxi Li et al.· 0 citations
This study theoretically shows that when zero-propensity samples exist, both IPS and DR estimators become biased, and proposes ExtraDebias method, which enables debiased recommendation in both non-extrapolation and extrapolation samples.
Yanghao Xiao, Hao Wang, Xiang Li et al.· Annual International ACM SIG...· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.