It is shown the path to generalizing L2 reasoning to held-out languages goes through broader language coverage, readily available multilingual non-reasoning data, and a sufficient English reasoning backbone, indicating that reasoning is a language-agnostic behavior that can be transferred across typologically diverse l...
Mehrnaz Mofakhami, Ananya Sahu, Alejandro R. Salamanca et al.· 0 citations
The IOL-AI Challenge is presented, an open-science competition run on the unseen problems of the International Linguistics Olympiad 2026 Individual Contest, evaluated both automatically and, for the first time, by members of the official IOL Jury under the same rubrics applied to human contestants.
Eduardo Sánchez, Rita Berrada, Dan-Mircea Mirea et al.· 0 citations
This work formalizes multi-model human evaluation as a best-arm identification problem in a multi-armed bandit setup with correlated arms, where pulling an arm corresponds to human-evaluating a model, and proves the optimality of the proposed algorithms and shows that it improves discrimination between top-performing m...
Vilém Zouhar, Julia Kreutzer, A. Lavie et al.· 0 citations
JudgeArena is an open-source framework that unifies major LLM-judge benchmarks under a single interface with swappable judges and comprehensive metadata logging for increased transparency in reporting and reproducibility and enables systematic studies of judge choices.
Erlis Lushtaku, Bora Kargi, Ali Elganzory et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
