Deep learning and computer vision framework for tomato insect classification with XAI insights

Abstract

Pest infestations pose a life-threatening challenge to global tomato production, often resulting in devastating yield losses and economic instability. Traditional manual identification practices are subjective, labor-intensive, and prone to error. To address this, within the paradigm of Agriculture 5.0, we introduce a novel hybrid deep learning model, ResNet50-Fusion (ViT), designed for the precise detection of six economically relevant tomato pests. The proposed architecture integrates a ResNet50 backbone with a Vision Transformer (ViT) branch via a Multi-Head Cross-Attention mechanism that facilitates asymmetric semantic alignment, enabling the simultaneous extraction of fine-grained morphological features and global contextual patterns. To ensure scientific rigor and eliminate the risk of data leakage, a ‘Split-then-Augment’ protocol was implemented, keeping the test set entirely independent and original. The framework achieved a state-of-the-art test accuracy of 97.00% ± 0.12%, significantly outperforming baseline models such as ResNet50 (96.44%) and DenseNet169 (96.00%), while demonstrating high computational efficiency with an average inference latency of 26.6 ms per image for real-time edge deployment. Beyond predictive accuracy, the model’s reliability was validated using a multi-method quantitative Explainable AI (XAI) framework integrating semantic indicators: Semantic Localization Score (SLS) for feature alignment, Robustness Score (RS) for logical stability, and Interpretability Reliability Coefficient (IRC) for decision consistency. Our results demonstrate a 96.5% Pointing Game Accuracy and a low Deletion AUC (0.142), indicating that the model’s attention patterns strongly align with biologically relevant morphological features rather than background artifacts. Systematic occlusion sensitivity analysis further confirmed model robustness with a stability score of 0.988. Finally, external validation on world-scale datasets, including IP102 and Pest24, yielded accuracies exceeding 90% via direct inference, demonstrating the generalizability of the framework to diverse automated field monitoring conditions. This research provides a precise, robust, and scalable solution for real-time pest identification in global precision agriculture.