Two-stage Transfer Learning for Chilli Leaf Disease Classification using CNN and Vision Transformer

Abstract

Chili cultivation is considerably affected by leaf diseases, making timely and reliable disease identification essential for maintaining agricultural productivity. While deep learning has demonstrated strong capabilities in plant disease recognition, many existing studies primarily focus on modifying network architectures rather than optimizing the training process. This study introduces a two-stage transfer learning framework that combines frozen-backbone adaptation with controlled fine-tuning to enhance classification performance. The experiments utilized a publicly available dataset comprising 12,000 augmented chili leaf images representing six disease categories. Several deep learning architectures were investigated, including DenseNet121, EfficientNetB0, MobileNetV2, ResNet152V2, and the Vision Transformer (ViT-B16). The experimental results indicate that the proposed two-stage training strategy improves classification performance consistently across the evaluated architectures. Among the models, ViT-B16 obtained the highest test accuracy of 99.61%, together with stable macro-averaged precision, recall, and F1-score values. Furthermore, the confusion matrix and ROC analysis revealed strong discrimination among the disease classes, with only limited errors occurring between visually similar categories. These findings highlight the importance of optimizing the training schedule when adapting pretrained models for plant disease recognition. Overall, the proposed framework provides an effective and scalable approach for chili leaf disease classification while offering an alternative to approaches that primarily rely on architectural modifications.