Do Language Models Know What Words Mean? Evaluating Semantic Knowledge of Adjective-Noun Compounds

Does Bielik Know What It Doesn't Know? Activation Dispersion Separates Entity Familiarity from Factual Reliability Across Model Scale

Large language models hallucinate most about entities they have never seen. We ask whether a model's activations betray entity familiarity before a single answer token is generated, and whether that signal predicts the factual reliability of the answers. On four Polish Bielik models (1.5B-11B parameters), we probe four...

How Much Human Label Variation Does Formal Semantic Structure Explain?: Group-Level Effects and Item-Level Ceilings in NLI

In this sample, formal semantic structure shifts how much annotators disagree by a small amount and does not detectably change what they disagree about, which suggests a group-level boundary and composition invariance.

Reversing Arrows in Large Language Models

This work presents the first systematic study of inverse relation directionality in LLMs, using a benchmark consisting of 5,457 instances spanning 27 distinct inverse relation labels and reveals systematic asymmetries in inverse relation classification across LLMs.

Predicting Human Semantic Priming Using Multilingual Large Language Models

Human-Like Anaphor Resolution in Large Language Models

The results show selective cognitive alignment: some LLMs exhibit human-like sensitivity to discourse prominence and distance-based factors in anaphor resolution, while showing weaker or absent sensitivity to semantic interference effects.

Large Language Models Under Evaluation: An Acceptability, Complexity And Coherence Assessment In Italian

The results suggest that, although fine-tuned transformers outperform all GPT models, GPT-4 represents a significant improvement over third-generation GPT models and poses a challenge to the poverty of the stimulus hypothesis.