Skip to content
Conference

ViePAWS: A Vietnamese Adversarial Dataset for Paraphrase Identification under LLM-based Word Scrambling

Aug 2026 · International Conference on Multimedia Analysis and Pattern Recognition · pp. 43-48 · 0 citations · 21 references

Abstract

Paraphrase identification remains challenging when sentence pairs exhibit high lexical overlap but subtle semantic differences, as models often rely on surface similarity rather than true meaning. Existing benchmarks such as PAWS highlight this issue, but comparable resources for Vietnamese are still lacking. In this paper, we introduce ViePAWS1, a Vietnamese benchmark for paraphrase identification under adversarial high-overlap conditions. The dataset is constructed to contain sentence pairs that are lexically and semantically similar while differing in meaning, making them difficult to distinguish using surface cues alone. We evaluate both pretrained models and large language models and observe consistent performance degradation compared to standard settings. In particular, models struggle to correctly classify non-paraphrase pairs with high similarity, revealing a strong bias toward lexical overlap. Our analysis further shows that different types of meaning variation lead to different failure patterns, indicating that current models lack robust mechanisms for capturing fine-grained semantic distinctions. These results highlight the need for more challenging evaluation benchmarks that better reflect true semantic understanding in paraphrase identification.

View source

Similar papers

Open access Sep 2026

English-Persian cross-language plagiarism detection: Multilingual deep learning approach

Cross-lingual plagiarism detection (CLPD) remains a critical challenge in Natural Language Processing, particularly for low-resource language pairs where translation and paraphrasing obscure source materials. This study introduces a robust framework for Persian-English CLPD by integrating state-of-the-art sentence repr...

Sara Ansar, M. Zarinbal, Azadeh Mohebi · 0 citations
#artificial intelligence Preprint Aug 2026

IndicDetect: Evaluating Cross-Lingual LLM-Generated Text Detection for Hindi, Telugu, and Tamil

IndicDetect provides standard data splits, an evaluation protocol, and baselines to establish a robust, language-aware foundation for AI-generated text detection in Indic scripts, and finds substantial robustness failures.

Bhaskar Ganesh Devalla, Junchao Wu, Nilesh Dokuparthi et al. · 0 citations
Preprint Aug 2026

Padamitra: Grounded Glossary Generation for Classical Sanskrit

We introduce grounded glossary generation, a structured task requiring models to recover semantically meaningful Sanskrit phrases and produce translation-grounded meanings from a sloka-translation pair, formalizing the traditional patha commentary practice as an evaluable NLP objective. We construct a benchmark of 31,3...

Manoj Balaji Jagadeeshan, Sai Pragnaan Marala, Pawan Goyal · 0 citations
Open access Oct 2026

Benchmarking fastText, word2vec, and BERT on the MAQASID Dataset: A Closed-Domain Religious Resource

In this paper, we introduce MAQASID, a closed-domain dataset for exploring the optimal semantic representation of Indonesian Islamic texts. It is designed to solve challenges in low-resource languages and closed-domain religious datasets, such as Arabic-to-Indonesian transliteration inconsistencies and semantic shifts....

Muhammad Zidny Naf'an, E. Winarko, S. Priyanta · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.