Identifying the impact of visual models on the quality of optical reading of information from food product packaging

Abstract

This study investigates automated extraction of structured food information (nutritional value, composition, serving size) from packaging images. The task addressed relates to the lack of a controlled comparison of small local visual language models (VLMs) with classical OCR (optical character recognition) systems on the domain task of extracting structured food information from packaging images under realistic print degradations. The most difficult is the low quality of retail panels: small font, regional variability of formats (EU "per 100 g" and US "per serving"), paint stains and glare, which suppress the effectiveness of regular expressions over linear OCR output. A reproducible benchmark has been built on 240 images (60 baseline × three Augraphy distortion modes) and 1200 calls of five models: Tesseract 5, PaddleOCR 2.10, Qwen2.5-VL-3B, Gemma-3-4B (in MLX-VLM format), and cloud-based Claude Haiku 4.5 as the outer boundary of quality; quality was assessed by macro-F1 on a single Apple Silicon laptop. Haiku 4.5 was found to achieve macro-F1 = 0.72 on clean images, 1.5 times better than the best small local model Gemma-3-4B (0.48); PaddleOCR is the most robust classical backend (7.5 points F1 reduction between best and worst conditions); Qwen2.5-VL-3B demonstrates a hallucination mode (completeness 0.74 at accuracy 0.25). The advantage of VLM is due to the joint processing of the image and the instruction while preserving the tabular structure of the panel, while classic OCR forms a linear flow in which the “nutrient-value-unit” connections are lost. The reproducibility of the protocol and the completeness of local execution make the recommendations applicable in retail audit systems, labeling control, and consumer applications; the results quantitatively characterize the cost of local execution and provide model selection rules for budget, quality, and local scenarios