Cost-Effective Entity Matching With Large Language Models and Diversity-Aware Example Selection

Abstract

Entity matching is the task of identifying database entries that refer to the same entity across multiple data sources, enabling data integration across domains. Recent studies have explored large language models (LLMs) for entity matching using in-context learning (ICL), where a small number of labeled examples are provided in the prompt. However, existing approaches implicitly assume the availability of large labeled datasets and do not consider the cost of annotation. In this paper, we propose a cost-effective LLM-based entity matching framework that reduces the number of annotations required for entity matching. The framework generates candidate pairs from input data sources using blocking and selects a small set of representative pairs for annotation. We introduce D-diversity, a diversity-aware example selection method that selects examples based on differences between entity pairs to improve ICL effectiveness. Experimental results show that diversity-aware selection generally improves average performance over random selection across multiple datasets. The proposed framework achieves competitive performance with only 10 annotated examples, reducing the number of annotated examples required by more than 97% compared with a deep learning-based method. Furthermore, we analyze the effects of varying the number of annotated examples, example ordering, and prompt design, and show that their impact varies depending on the model and dataset. These findings provide practical insights for applying LLM-based entity matching under limited annotation budgets.