TL;DR
A data-centric analysis of semantic knowledge acquisition in word embeddings, focusing on word analogy and semantic similarity shows that, for relational semantics, training-data quality outweighs quantity, and that simple proxy models remain a practical, interpretable tool for efficient data selection.
The proposed approach provides a language-agnostic, context-sensitive segmentation mechanism that can complement language processing methods such as lemmatization, morphological analysis, and stemming and indicate task-dependent and generally limited improvements over traditional token-based word-embedding extraction.
LMEnt is released to support studies of knowledge in LMs, including knowledge representations, plasticity, editing, attribution, hallucinations, and learning dynamics, finding that entity co-occurrence and mention forms—which are difficult to study with existing tools—affect learning trends.
Protein language models (PLMs) have transferred the latest advances from natural language processing to computational biology. These models, trained on large corpora of protein sequence data, are widely used to translate amino acid sequences into latent-space embeddings, ready for use in diverse downstream tasks (DTs)....
On a large-scale all-words WSD task, the encoder model not only outperformed the decoder model but also generated substantially lower carbon emissions – an eight-fold reduction.
The findings suggest that vocabulary development can be understood through the non-trivial interplay between activation dynamics and some degree of constraints regulating the visiting of lexical categories in complex networks.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
