The results show that models’ verb representations degrade more when the authors perturb word order, which removes syntactic information, than when they perturb co-occurrence among content words, which removes semantic information.
Abstract
Syntactic bootstrapping (Gleitman, 1990) is the hypothesis that children use the syntactic environments in which a verb occurs to learn its meaning. Existing evidence for this hypothesis generally involves controlled experimental settings (e.g., Jin & Fisher, 2014; Naigles, 1990; Yuan et al., 2012). In this paper, we leverage the powerful statistical learning abilities of neural language models to examine the contribution of different kinds of cues for word learning when a learner is presented with a corpus of English linguistic input on the scale of a child’s experience. We do this by training two standard types of neural networks (masked and autoregressive transformers) on perturbed datasets where different sources of information relevant to word learning are individually removed. Our results show that models’ verb representations degrade more when we perturb word order, which removes syntactic information, than when we perturb co-occurrence among content words, which removes semantic information. Furthermore, the representations of mental verbs, for which syntactic bootstrapping has been shown to be particularly crucial in human verb learning, are more negatively impacted in the training regime with perturbed word order as compared to physical verbs. In contrast, while word order also affects models’ representations of nouns, the nouns are influenced more by co-occurrence information than the verbs are. In addition to reinforcing the important role of syntactic bootstrapping in verb learning, our results demonstrate the viability of testing developmental hypotheses on a large scale through manipulating the learning environments of large language models.
Acquiring telicity, which is the distinction between bounded (e.g., ate an apple) and unbounded (e.g., ate apples) events, requires first language (L1) learners to map surface-level and semantic cues to abstract event structures, but the computational trajectory of this mapping is not well understood. We introduce a Di...
Ellie Xia, Parisa Kordjamshidi, Alan Hezao Ke· 0 citations
Large language models learn word meanings through distributional semantics (i.e., patterns in text), and distribution does not always capture information about taxonomy and function, which are essential to human representations of word meaning. However, sound and spelling often conveys such information, and language...
David A. Haslett, Antoni B. Chan, J. Hsiao· Transactions of the Associat...· 0 citations
Abstract Natural languages avoid syntactic structures with crossed syntactic dependencies. To the extent that such structures nonetheless occur, they have been explained as the result of competing motivations, such as dependency length minimization, the facilitation of incremental language processing, or the preservati...
Martin Hilpert· Cognitive Linguistics· 0 citations
SLITE is presented, an explainable hybrid model for Recognizing Textual Entailment that integrates two complementary layers of semantic analysis: a structural-relational layer, based on semantic compatibility and incompatibility between compositional entities, and a distributional-informational layer, based on structur...
David Torres-Moreno, J. Hermosillo-Valadez, Asela Reig-Alamillo· arXiv.org· 0 citations
A two-layer LSTM language model trained on 250M morphemes of Hindi data can capture the relatively complex interaction between case and agreement in Hindi-Urdu, and it is shown that this model encodes case-marking linearly, implementing a geometrically intuitive and inter-pretable syntactic processing mechanism.
Satoru Ozaki, Rajesh Bhatt, Brian Dillon· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.