Food Recognition and Calorie Estimation Using YOLO
TL;DR
This paper presents a real-time food recognition and calorie estimation system that leverages the YOLOv8 nano object-detection architecture coupled with a curated nutritional knowledge base and outperforms prior approaches based on Faster R-CNN and SSD MobileNet on both speed and accuracy metrics.
Abstract
Monitoring dietary intake is a cornerstone of chronic disease management and healthy lifestyle promotion, yet it remains a persistently manual and error-prone process. This paper presents a real-time food recognition and calorie estimation system that leverages the YOLOv8 nano object-detection architecture coupled with a curated nutritional knowledge base. The proposed system is capable of identifying multiple food items in a single image or live webcam stream and immediately associating each detection with an estimated caloric value. We fine-tuned a pre-trained YOLOv8n model on a custom food dataset of 48 classes drawn from both Western and Indian cuisines and integrated the resulting detector with a Gradio-based web interface for accessible deployment. Experimental results demonstrate a mean average precision (mAP@0.5) of 91.7% on the held-out test set, an inference speed of 87 frames per second on an NVIDIA T4 GPU, and a mean absolute error (MAE) of 16.8 kcal per item across the test categories. The system outperforms prior approaches based on Faster R-CNN and SSD MobileNet on both speed and accuracy metrics. An open-source implementation is made available alongside a demonstration video, positioning this work as a practical foundation for mobile diet-tracking applications and clinical nutrition monitoring tools.