The study resulted in the development of a novel six-component framework comprising Input Processing, LLM Core, Knowledge Enhancement, Context Management, Response Generation, Response Generation, and Human Feedback that successfully addressed resource scarcity through language detection and cross-lingual query understanding.
CALAMITA is conceived as a rolling benchmark, enabling continuous integration of new tasks and models, and argues that this combination offers a blueprint for other languages and communities seeking inclusive and rigorous LLM evaluation practices.
Malvina Nissim, Danilo Croce, V. Patti et al.· Italian Journal of Computati...· 0 citations
Whether contemporary LLMs can reproduce the research outcomes of a fully documented human study: a 1991 article that identified dermatophytosis (ringworm) in historical fine art was evaluated.
The research confirms that ChatGPT successfully identifies functional equivalents in the Serbian language, often prioritising the naturalness of expressions over literal translation, often prioritising the naturalness of expressions over literal translation.
Jelena Janackovic, Jovana Bošković, Jelena Mladenović· SINTEZA· 0 citations
A multi-layer taxonomy of 14 capability domains and 91 subskills across Primitive, Constructed, and Integrative layers is introduced and supports research organization, coverage audits, evaluation interpretation, and testable hypotheses for diagnosis, training, and transfer.
A novel, layered conceptual framework is introduced that organizes research in CDR across four key dimensions: User Layer, System Layer, Data Layer, and Evaluation Layer and identifies core challenges in CDR, including the lack of standardized evaluation benchmarks and limited support for ambiguous or evolving user intent.
Lisa-Yao Gan, Johanna Walker, E. Simperl et al.· Information Systems Frontier...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
