A Method for Dynamic Expansion of Domain Dictionaries and Weakly Labeled Named Entity Recognition for Low-Resource Specialized Corpora

Abstract

In order to solve the problems of domain entity lack and high noise, little human supervision in low resource specialized corpora, this work adopts the method of multi-feature fusion of dynamic domain dictionary expansion. The method includes dictionary matching, semantic confidence filtering and confidence-weighted named entity recognition, and realizes the mutual iteration between the dictionary and the recognition model by feeding back the entity. The results demonstrate that, the dynamic expansion raised the entity coverage from 38.10% to 77.22% while the accuracy of adding new entities is 94.67% for the dynamic expansion; the F1 score of the confidence based filtering for weakly labelled data is 84.68%, and the Micro-F1 score of the complete model is 89.17%, which is 13.28 percentage points better than baseline model under 1% human annotation. It shows that the approach proposed in this study is able to complement the long tail specific entities while keeping noise from weak labels under control and the entity recognition performance still good under low level manual annotation.