Research on Qiaopi Character Recognition Based on Large Models

Abstract

Significant progress has been made in recognizing handwritten text in images using computer vision techniques. However, recognizing text within Qiaopi images still presents considerable challenges due to the inherently handwritten nature of Qiaopi text, often compounded by issues such as illegible handwriting, aging paper, ink diffusion, background noise, and irregular layouts. In recent years, large model technology has advanced rapidly. To evaluate the ultimate capabilities of large models in recognizing text within Qiaopi images, this paper constructs the first benchmark dataset specifically for text recognition in Qiaopi images. The text recognition capabilities of multiple large models were evaluated; an algorithm for integrating multiple large models based on multiple sequence alignment was proposed; furthermore, an algorithm was developed to further correct errors in the integrated results using a large language model; additionally, an algorithm was proposed that utilizes the large model ERNIE5.0 to implement the integration of output results from multiple large models. Experimental outcomes reveal that employing a single large model for Qiaopi text recognition can achieve maximum AR and CR metric values of 83.71% and 84.81%, respectively. In contrast, the algorithm incorporating multi-sequence alignment integration and large model error correction yields AR and CR metric values of 84.73% and 86.45%, respectively.