It is demonstrated that Big Five inventories do not measure a construct equivalent to human personality in LLMs, highlighting the need for evaluation frameworks that are specifically designed and validated for LLMs, rather than transferring human psychological constructs without first establishing their validity.
Kim Zierahn, Cristina Cachero, Anna Korhonen et al.· arXiv.org· 0 citations
Overall, the results show that HLV can be learned as annotator-specific label-explanation behavior, suggesting a path toward scalable explanation-based annotation grounded in annotator histories rather than labels alone.
LLM evaluators (trained reward models and prompted LLM-as-a-Judge) are routinely validated via pairwise accuracy. In a multilingual setting, this operates under the premise that high pairwise accuracy implies reliable, language-neutral scoring. We show that this assumption does not hold. We conduct experiments with sem...
Yijie Zhou, Lucas Resck, Zheng Hui et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.