A hybrid deep learning–based OCR model for handwritten medical prescriptions

Abstract

The current trend involving the use of paper-based medical prescriptions and the exchange of such by the social media is a major problem in automated and accurate text extraction in healthcare processes. This paper suggests a complete smart pipeline to digitize the images of medical prescriptions with lightweight deep learning-based classification, specialized Optical Character Recognition (OCR) engines, and comparative benchmarking of results using a large language model (LLM). The web scraped images of medical prescriptions are initially processed through a GhostNetV2 classifier to differentiate between handwritten and digital medical prescriptions. According to this choice, digital images are sent to PaddleOCR, and handwritten images are considered in Tesseract OCR so that each image could be processed by the best suitable recognition engine. The OCR outputs from both branches are benchmarked against GPT-4o (OpenAI) as an independent baseline to assess competitive performance and provide error-rate context. Experimental analysis shows that the suggested adaptive routing strategy is characterized by the fact that it has a high classification rate and the character and word error rates are significantly lower than those single-OCR techniques. The system is competitive with, though does not exceed, the raw accuracy of a state-of-the-art LLM-based OCR baseline (GPT-4o), while offering practical advantages in cost, transparency, and on-premises deployability that the black-box, per-call LLM baseline does not. The findings verify that the combination of lightweight classifiers and multi-OCR decision routing and multi-LLM-aided comparative benchmarking offers a scalable and robust solution to real-world medical prescriptions digitization, which adds to better automation, accuracy, and safety in healthcare information processing.