A Lightweight One-Shot Open-Set Metric Learning Framework for Food Recognition and Decision Support in Smart Ovens

Abstract

Modern smart kitchen automation requires reliable vision-based tools to provide user-advisory decision support during domestic culinary processes. However, standard deep learning models utilizing closed-set Softmax classifiers typically misclassify unknown or Out-of-Distribution (OOD) kitchen objects with high confidence, posing safety and reliability risks. To address this problem without clous dependency, this study introduces a localized open-set metric learning framework based on a modified MobileNetV2 architecture. The conventional Softmax classification layer is replaced with a feature embedding layer evaluated via Cosine Similarity and a calibrated decision threshold. This architecture tracks targeted food items across four operational stages—counter-raw, in-oven-raw, in-oven-cooked, and counter-cooked—while identifying and rejecting OOD objects. To ensure reproducibility, comprehensive experimental validations were conducted on a dedicated internal dataset, providing direct baseline comparisons against mainstream backbones (ResNet50, EfficientNet-B0, and Vision Transformers) stripped of their Softmax layers and evaluated under identical metric constraints. The results demonstrate that the proposed framework achieves a Macro F1-score 92.6% and ultra-low inference latency of 11.5 ms, ensuring an optimized trade-off between Macro F1-score and inference speed on edge computing environments. This framework establishes a robust, self-contained solution for open-set object recognition in localized smart home appliances.