AgrySET: A Synthetic Data Generation Framework for Agricultural Semantic Segmentation and Defect Recognition

Abstract

While existing synthetic generation methods, such as Generative Adversarial Networks (GANs) and Diffusion Models, typically require large pre-existing datasets and substantial computational resources, this paper presents AgrySET: a novel hardware–software framework that bypasses these limitations. The system utilizes a four-camera high-resolution array and a setup that replicates industrial environmental conditions. Unlike purely algorithmic approaches, by employing Gaussian models for data generation on physically acquired samples, the framework actively addresses class imbalance issues and synthesizes realistic agricultural imagery alongside exact, pixel-perfect segmentation masks without manual pixel-level annotation after initial sample acquisition. As a case study, the segmentation of healthy coffee beans and seven defects as categorized by the National Federation of Coffee Growers of Colombia was performed. Models trained exclusively on this synthetic dataset achieved a Mean Intersection over Union (mIoU) of 92.9% in a 3-class configuration and 61.9% in a detailed 9-class scenario. When evaluated against real-world datasets, the model yielded accuracies of 90% and 58%, respectively. Furthermore, the automated pipeline accelerated the data generation and segmentation workflow by an estimated factor of 250 compared to manual execution. These quantitative results demonstrate the feasibility of AgrySET as a baseline synthetic framework for coffee bean inspection, highlighting its potential for future adaptation to other granular agricultural commodities.