Preslav Nakov

SQLStructEval: Structural Evaluation of LLM Text-to-SQL Generation

SQLStructEval is introduced, a framework that analyzes this behavior through canonical abstract syntax tree representations and adopts a pipeline that first generates structured intermediate representations and then deterministically compiles them into SQL, improving execution accuracy and structural agreement among co...

MIC: Explaining Image-Claim Inconsistencies in AI-Generated Multimodal Misinformation

Claims paired with AI-generated images are a rapidly growing form of misinformation. Existing automated fact-checking (AFC) methods mainly treat this as a provenance problem, detecting low-level synthesis artifacts to decide whether an image is AI-generated. However, such methods do not verify what human fact-checkers...

Same Scores, Different Decisions: Evaluating JEV and Language Models for Legal Document Understanding

Jev has the lowest cost and median response time among the evaluated configurations, while hosted language models achieve higher baseline accuracy and Rankings by baseline accuracy differ from rankings by correctness across every condition and repeat, although small differences in the latter do not establish a general...

Co-FactChecker: A Framework for Human-AI Collaborative Claim Verification Using Large Reasoning Models

Co-FactChecker is proposed, a framework for human-AI collaborative claim verification that translates expert feedback into trace-edits that introduce targeted modifications to the trace, sidestepping the shortcomings of dialogue-based interaction.

MemeLens: Multilingual Multitask VLMs for Memes

It is suggested that robust meme understanding requires multimodal training, varies substantially across semantic categories, and remains sensitive to over-specialization when models are fine-tuned on individual datasets rather than trained in a unified setting.

Detecting AI-Generated Bulgarian Text: A Two-Step Multi-Class Classification Approach

This paper introduces a novel two-step multi-class classification system to identify varying degrees of machine involvement in Bulgarian text. As Large Language Models (LLMs) proliferate, distinguishing original human writing from machine-assisted or machine-generated content is crucial to prevent misinformation and pr...

Cultural Benchmarking of LLMs in MSA and Arabic Dialectal Dialogue

FinCUABuild: Can Agents Build Reliable Benchmarks for Dynamic Financial Computer Use?

This work introduces FinCUABuildBench, a benchmark for evaluating financial CUA task construction, and introduces FinCUABuildAgent, a multi-agent system for automatically constructing dynamic financial CUA evaluation tasks.

Can Dialects Be Steered Like Languages? Sparse Neurons and Distributed Directions in Arabic LLMs

Neuron steering reinforces dialect in some varieties when the prompt is already dialectal but cannot induce it from MSA prompts, whereas vector steering succeeds in both settings, and Arabic dialects are therefore steerable mainly through distributed rather than localized representations.

Jais 2: A Family of Arabic-Centric Open Large Language Models

Jais 2 is a family of Arabic-centric large language models developed jointly by MBZUAI, Cerebras, and Inception, designed to advance Arabic-centric language modeling, with strong performance across the Arabic and culturally grounded benchmarks evaluated in this report. The family includes, to our knowledge, the largest...

How Correct Is Your Answer? A Semantic Correctness Framework for Open QA Evaluation

A semantic correctness taxonomy is introduced that assigns open-ended answers to eight ordered classes, separating verbose-but-correct answers from those contaminated by hallucinated content and CAP (Context-Aware Precision), a reference-based metric that scores question-conditioned statements using bidirectional NLI.

EDRAC: Benchmarking Arabic Dialect Reading Comprehension

This work introduces EDRAC, the first large-scale benchmark for dialectal Arabic machine reading comprehension (MRC) and generative QA, covering five major dialects: Egyptian, Moroccan, Emirati, Syrian, and Saudi Arabic, and benchmarks Arabic-centric and multilingual LLMs on EDRAC using lexical and semantic metrics.