Grouping tokens in a balanced linguistic corpus based on semantic criteria

Abstract

The aim of the study is to develop a procedure for grouping tokens from a balanced linguistic corpus based on their lexical meaning. The scientific novelty lies in the first-time fully automated testing – using the “Balanced Linguistic Corpus Generator and Corpus Manager” software package – of a technology for identifying the meanings of lexical units; this establishes principles for constructing semantic series by leveraging managed artificial intelligence tools within a corpus-based framework. As a result, software code was written and integrated into the corpus manager, enabling the automatic generation of over 1,700 relevant noun groups (ranging from 2 to 174 units), over 650 adjective groups (from 2 to 79 units), and over 460 verb groups (from 2 to 12 units) from a German-language news corpus comprising about 4.18 million tokens.