This study quantitatively evaluates how syntactic and semantic information affects word-grounding performance and demonstrates the contribution of linguistic structure and semantic representations to perceptual alignment.
Abstract
Introduction Word grounding refers to the ability of agents to associate linguistic terms with the perceptual concepts they represent, such as color, shape, and spatial features. This study quantitatively evaluates how syntactic and semantic information affects word-grounding performance. Methods We evaluated five configurations within a generative multimodal Bayesian framework. The baseline grounds words using perceptual cues, while the additional configurations independently incorporate word-position indices, part-of-speech tags, FastText embeddings, and transformer-based contextual embeddings. The models were evaluated using a CLEVR-derived 3D scene dataset and synonym-substituted descriptions. Results Linguistic information improved grounding accuracy across the Color, Geometry, and Spatial Relation modalities. Contextual embeddings produced a 53.8% absolute improvement in object-grounding accuracy over the baseline. Only models incorporating semantic embeddings generalized effectively to synonym-substituted descriptions without retraining. Discussion These findings demonstrate the contribution of linguistic structure and semantic representations to perceptual alignment and provide empirical foundations for developing language-grounding systems.
SLITE is presented, an explainable hybrid model for Recognizing Textual Entailment that integrates two complementary layers of semantic analysis: a structural-relational layer, based on semantic compatibility and incompatibility between compositional entities, and a distributional-informational layer, based on structur...
David Torres-Moreno, J. Hermosillo-Valadez, Asela Reig-Alamillo· arXiv.org· 0 citations
This work designs a semantic structure-based feature refinement module to adaptively modulate subject-centric and contextual visual features and incorporates a visual semantic consistency verification module, which leverages a subject-aware contrastive learning strategy to constrain and verify the semantic corresponden...
Jie-Peng Cai, Zhen Xu, Tie-Song Zhao et al.· Proceedings of the Thirty-Fi...· 0 citations
A layer-wise analysis indicates that surface-level features such as temporality and negation are captured more reliably than deeper semantic phenomena like quantification in large language models, highlighting the limited capacity of current LLMs to generate fully formal meaning representations.
Rémi De Vergnette, Maxime Amblard· International Conference on...· 0 citations
Large language models learn word meanings through distributional semantics (i.e., patterns in text), and distribution does not always capture information about taxonomy and function, which are essential to human representations of word meaning. However, sound and spelling often conveys such information, and language...
David A. Haslett, Antoni B. Chan, J. Hsiao· Transactions of the Associat...· 0 citations
This paper revisits semantic projections and related count-based representations as interpretable directional semantic structures for semantic analysis in document corpora and web-based information environments and demonstrates that semantic projections effectively capture persistent contextual structures while remaini...
Mabel López-Bordao, Antonia Ferrer-Sapena, Pablo Lara-Navarra et al.· Information· 0 citations
One of the fundamental issues in the cognitive sciences concerns the question of how language attains meaning. The current study investigated how the extent to which language statistics and symbol grounding reduce uncertainty about word meaning can be quantified. Using Shannon’s information theory, conditional entropy...
M. Louwerse· Psychonomic Bulletin & Revie...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.