A controlled evaluation framework for multilingual natural language inference that stress-tests semantic consistency using synthetic logic-based premise–hypothesis pairs in both monolingual and code-switched conditions is presented.
Multilingual large language models often struggle to reason in low- to mid-resource languages. Prior work has shown that translation can improve multilingual reasoning by helping models access stronger English-centric representations. This raises a central question: How much translation is needed for multilingual large...
Standard factual benchmarks assume clean monolingual prompts and exact-match scoring, two assumptions that break simultaneously in real-world multilingual deployment, from retrieval-augmented generation pipelines returning mixed-language passages to users pasting multilingual web content. We introduce Multilingual Dist...
Riju Marwah, Ritvik Garimella, Khusham Bansal et al.· 0 citations
This work presents a unified evaluation of representative CLC-enhancement methods for question answering, spanning inference-time interventions and post-training approaches across three model families and three closed-form benchmarks, highlighting the need to evaluate CLC enhancement for both cross-domain robustness an...
Jirui Qi, Ming-Yang Wang, Hinrich Schütze et al.· 0 citations
It is argued that collapsing cultural competence into a single scalar incentivizes "probabilistic flattening," and that recognizing the unalignable class is a precondition for AI that respects, rather than erases, cultural divergence.
Multilingual large language models (LLMs) are widely assumed to behave consistently regardless of the language in which a prompt is expressed. We challenge this assumption through a controlled empirical study in which language itself is isolated as the sole independent variable. We present 30 semantically equivalent pr...
Aman Chandra H, Sameer Kulkarni, S. Nithin et al.· 2026 International Conferenc...· 0 citations
A novel knowledge graph-based evaluation framework is proposed introducing Semantic Structural Similarity for KGs (S3KG), a hybrid similarity measure integrating structural and semantic similarity into a continuous evaluation score, alongside a diagnostic framework for categorizing reasoning errors.
Subavarshana Arumugam, Mamta Nallaretnam, K. Wickramasinghe et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
