Balancing accuracy and efficiency: Evaluating encoder- and decoder-based models for word sense disambiguation and regular polysemy detection
TL;DR
On a large-scale all-words WSD task, the encoder model not only outperformed the decoder model but also generated substantially lower carbon emissions – an eight-fold reduction.
Abstract
This study investigates the nuanced challenges of fine-grained word sense disambiguation (WSD) tasks with regular polysemy detection (RPD) of the named entity, focusing on evaluating the trade-offs between encoder and decoder-based model performance and computational efficiency. The datasets, including Chinese Wordnet 2.0 (CWN) as sense inventory, the Social Media Corpus (PTT) for user-generated content, and the Academia Sinica Balanced Corpus (ASBC) for formal linguistic data, were chosen to provide a diverse and representative framework for evaluating both common nouns and proper nouns with regular polysemy in Taiwan Mandarin. This analysis evaluated ten encoder- and decoder-based models, assessing their performance on two tasks. The encoder-based models demonstrate comparable accuracy to the decoder-based models on WSD tasks (77.5% vs. 78.5%), and similarly strong performance in RPD tasks (84.2% vs. 83.8%). On a large-scale all-words WSD task, the encoder model not only outperformed the decoder model but also generated substantially lower carbon emissions – an eight-fold reduction. These differences underscore the trade-offs between model architecture and task-specific performance, highlighting the necessity for balancing performance and energy efficiency in the design and application of language models, advocating for sustainable and eco-friendly practices in natural language processing development.