Systematic Review as Recipe Book for Nuanced Emotion Detection in Noisy Text

Abstract

Recognizing emotion in informal text at the granularity mental health support requires, telling nervousness from fear or remorse from sadness, remains far below usable accuracy: five years of work on Google’s GoEmotions multi-label 28-emotion dataset has reached only .5475 macro F1, and no systematic review has examined why. Accurate emotion recognition is an important predictor of mental healthcare success, yet its subjectivity makes it difficult for even trained human annotators to agree on nuanced emotions in noisy text. Our systematic search identified 494 papers. Inclusion required the full 28-emotion dataset, deep learning models, and reported macro F1, yielding 30 papers and 71 models. Among the highest-scoring models, 9 of the top 10 employ explicit emotion-learning architectures or objective functions. We found no LLM-based models exceeding the foundation paper’s baseline performance. Only 5 of the 71 models underwent extensive hyperparameter tuning. We attributed the observed ceiling to tuning practice as much as architectural limits. Systematic reviews are rare in AI/ML relative to medicine. This scarcity, combined with a submission surge that has outpaced peer-review capacity, plausibly explains why the field’s true state-of-the-art result went unrecognized for years. This recipe book details the emotion-specific design and tuning techniques associated with the strongest results. The strongest results in the corpus are associated with combining explicit emotion-learning architectures, objective functions, and rigorous tuning, though the reviewed models remain well short of the performance likely needed for practical usability in mental healthcare. Current evidence does not support LLMs as a competitive alternative.