It is found that providing candidate senses, similar to what is done in traditional WSD, improves WiC performance in all settings and helps models make more consistent and targeted judgements.
Abstract
Word-in-Context (WiC) remains challenging for language models, despite recent progress on lexical-semantic tasks. We hypothesise that this difficulty arises not only from comparing two contextual uses of a word, but also from the absence of an explicit sense inventory that specifies the relevant level of semantic granularity. We evaluate open LLMs on WiC and traditional Word Sense Disambiguation (WSD) under similar settings. We find that providing candidate senses, similar to what is done in traditional WSD, improves WiC performance in all settings. In general, explicit sense information helps models make more consistent and targeted judgements. Human evaluation further shows that many apparent WiC errors reflect label ambiguity or mismatches between model and annotator sense boundaries rather than simple failures of lexical understanding. In particular, results show that LLMs overthink the sense distinction often leading to errors based on overly fine-grained distinctions.
Findings highlight the complementary nature of WiC and WSD and demonstrate that unified training strategies can yield more robust and generalizable sense disambiguation models, and provide practical guidance for designing datasets and models in multilingual and low-resource contexts.
Alp Mujko, Dominik Schlechtweg· International Conference on...· 1 citation
LLM-assisted sense assignment with a Serbian WordNet-based custom inventory, iterative inventory expansion, and expert validation is combined with a constrained JSON-formatted output to support the practical construction and refinement of sense-annotated resources in a low-resource setting.
Saša Petalinkar, R. Stanković, Milica Ikonić Nešić et al.· Intelligent Data Analysis· 0 citations
The results support shifting ASR evaluation toward CER both for English and for morphosyllabic writing systems as it is a more interpretable and low-cost metric for what evaluation should actually capture, and using SemDist as a complementary evaluation.
How faithfulness of LLMs to provided context depends on how plausible they perceive the context to be (context-memory conflict) is analyzed, which makes use of the increased difficulty of non-English and low-resource language text generation and input data based on local knowledge.
Peter Kochelka, Aleš Manuel Papáček, Vojtěch Dvořák et al.· 0 citations
Inspicio is presented, an open-vocabulary retrieval pipeline that links tokens in context to synsets of the Open English WordNet (McCrae et al., 2020) without requiring any source-language inventory or mapping.