Vision-Based Automated Inspection of Box Meals for Food Portion Defects and Foreign Object Detection

Abstract

Automated inspection of prepared meals is important for improving food safety, quality assurance, and production efficiency. However, vision-based inspection remains challenging because boxed meals contain multiple adjacent food items with irregular shapes, varying portion sizes, and visually similar appearances, while oily surfaces may introduce specular reflections that degrade image quality. This study presents a vision-based framework for multi-object recognition and quantitative defect analysis in complex meal images, with Chinese-style lunch boxes used as representative test samples. The framework integrates region-of-interest (ROI) extraction, fixed-grid regional feature representation, deep neural network (DNN) classification, flood-filling post-processing, and empirical food-quantity thresholds. The lunch-box ROI is first extracted using the Hough transform, followed by median filtering to suppress reflection noise. The ROI is partitioned into 6 × 6 regions, from which the mean and standard deviation of RGB, HSV, and CIE Lab* color components are extracted and classified using a DNN. Flood filling is subsequently applied to refine the classification results, and category-specific empirical thresholds are used to identify missing food items and insufficient portions. Foreign objects are detected as an additional abnormal category. Under the evaluated experimental conditions, the proposed framework achieved an overall image classification rate (CR) of 96.45%, a defective-image detection rate (1−β) of 97.76%, a normal-image false alarm rate (α) of 5.34%, and a defective-image misclassification rate (γ) of 0.82%. Sensitivity experiments involving illumination variation, two lunch-box configurations with different food compositions, and conveyor-based image acquisition further demonstrated the feasibility and stability of the framework under the evaluated laboratory and prototype conditions. These findings support the feasibility of the proposed lightweight framework for vision-based quality inspection of representative boxed meals, while broader validation across meal types, contaminants, and industrial production environments remains necessary.