The paper introduces the Bulgarian benchmark Reasoning-BG, designed to evaluate how accurately large language models (LLMs) solve Bulgarian long-text comprehension tasks involving different comprehension demands. It was developed using a hybrid methodology that combines automated generation of four-option multiple-choi...
Maria Andreeva Todorova, Valentina Stefanova, T. Dimitrova et al.· Proceedings of the ... Inter...· 0 citations
Assessing the broad general knowledge of Large Language Models (LLMs) across multiple domains in Bulgarian remains challenging due to the limited availability of Bulgarian evaluation benchmarks. To address this gap, we introduce the Bulgarian Massive Multitask Language Understanding benchmark ( MMLU-BG ), designed to e...
S. Koeva, I. Stoyanova, Dimiter Georgiev et al.· International Conference on...· 1 citation
The dataset BulMMLU:gr, modelled on the MMLU (Massive Multitask Language Understanding) benchmark, is presented, to test the ability of AI grammar-checking applications and different LLMs to detect grammatical errors in Bulgarian and thus provide a basis for analysing the need for a specialised LLM-based tool to detect...
S. Koeva· Proceedings of the ... Inter...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.