Skip to content
Open access

Multimodal Contextual Learning for Maize Disease Diagnosis: Fusing Visual and Environmental Data

Jul 2026 · Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) · Vol 10, pp. 937-948 · 0 citations

TL;DR

It is demonstrated that integrating environmental context with visual symptoms enhances diagnostic accuracy and model interpretability, establishing a foundational proof-of-concept for context-aware agricultural AI.

Abstract

Maize Lethal Necrosis (MLN) and Maize Streak Virus (MSV) threaten food security in Zimbabwe, yet traditional image-based deep learning models lack adaptability to varying agro-environmental conditions. This study presents a multimodal contextual learning framework that fuses visual symptoms with environmental data to improve diagnostic accuracy and interpretability. A multimodal Convolutional Neural Network integrates leaf imagery with six environmental parameters (temperature, humidity, rainfall, soil moisture, leafhopper count, days after planting) using a late-fusion architecture with multi-task learning for simultaneous disease classification and five-stage severity estimation, enhanced by Gradient-weighted Class Activation Mapping (Grad-CAM) for visual explanation. A synthesized dataset (n=9,356) representative of Zimbabwean conditions was used. The multimodal approach achieved 94.3% accuracy versus 91.5% for image-only baselines (p<0.001, 95% CI), a 33.2% relative error reduction, with MSV-MLN confusion decreasing by 40.7% (113 to 67 misclassifications). Multi-task performance reached 94.5% for classification and 81.5% for severity estimation (F1=0.805). Grad-CAM analysis revealed environmental integration enhanced attention by 38.1% and localization by 45.2%, with leafhopper count showing the strongest correlation (r=0.62). Ablation studies confirmed that environmental features provided the largest accuracy gain (+3.71%, p<0.001). This work demonstrates that integrating environmental context with visual symptoms enhances diagnostic accuracy and model interpretability, establishing a foundational proof-of-concept for context-aware agricultural AI.

Read PDF

Similar papers

Review Open access Aug 2026

Multimodal Deep Learning and Foundation Models for Early Detection and Forecasting of Plant Diseases

A decade of progress across four interconnected frontiers is synthesizes the evolution of deep learning architectures for plant disease detection, the adaptation of foundation models including CLIP, SAM, and DINOv2 to agricultural contexts, and the development of multimodal fusion frameworks integrating imagery, environmental, genomic, and hyperspectral data are synthesized.

Teja Manda, Tian-Yu Huang, Yifan Ding et al. · 1 citation
Open access Jul 2026

AgriFusionNet: a context-aware multimodal leaf disease diagnosis and classification system for sustainable plant health monitoring

AgriFusionNet is discussed, a context-aware multimodal deep learning framework that integrates leaf images, textual symptom descriptions, and environmental data for robust plant disease classification and facilitates the co-learning of visual, semantic, and contextual environmental representations.

V. C., Nischith N Shetty, M. Ramaiah et al. · 0 citations
Sep 2026

Climate-Aware Multimodal Deep Learning Framework for Banana Leaf Disease Recognition Using CNN-Based Image and Environmental Feature Fusion

Banana cultivation in Maharashtra, particularly in the Jalgaon district, is severely affected by foliar diseases that significantly reduce crop productivity and quality. Traditional disease diagnosis methods rely heavily on manual inspection and expert intervention, which are often limited in rural agricultural regions. Moreover, existing deep learning-based disease detection systems primarily depend on visual image features while overlooking the influence of environmental conditions on disease progression. This study aimed to develop a climate-aware intelligent framework for accurate and context-sensitive banana leaf disease identification. The proposed framework integrates convolutional neural network (CNN)-based image analysis with agro-climatic parameters, including temperature, humidity and precipitation. A region-specific dataset containing approximately 18,000 banana leaf images representing nine disease categories was developed using field-level image acquisition and augmentation techniques. In parallel, long-term meteorological data collected from the Jalgaon region were incorporated to capture environmental conditions associated with disease occurrence. The CNN-extracted visual feature vector was fused with climate feature vectors and subsequently classified using fully connected layers and a SoftMax classifier. Experimental results demonstrated that integrating climate intelligence with image-based deep learning significantly improved disease classification performance. The proposed multimodal framework achieved an overall classification accuracy of 99%, outperforming the conventional image-only CNN models. The inclusion of climate features improved precision, recall and overall robustness by enabling the model to learn correlations between environmental conditions and disease patterns. The framework also demonstrated strong capability in distinguishing visually similar diseases under varying field conditions. Unlike the conventional image-centric approaches, the proposed framework introduced an environmentally contextualized disease recognition model, integrating CNN derived visual features with regional climate information, capable of supporting early warning systems and precision agriculture applications.

Unknown authors · 0 citations
Open access Jul 2026

MultiCotNet: a novel multispatial attention-based deep learning architecture for cotton leaf disease classification

The proposed MultiCotNet framework provides a scalable and reliable solution for early cotton disease detection and can support intelligent agricultural monitoring systems for timely disease management and improved crop productivity.

Sabari Nathan, S. A., K. S et al. · 0 citations
Open access Sep 2026

Deep convolutional neural network-based automated identification and classification of mungbean foliar diseases

Early and accurate detection of plant diseases is critical in precision agriculture to improve crop management and yield. Mungbean ( Vigna radiata L.) is highly susceptible to several foliar diseases, including yellow mosaic, powdery mildew, leaf crinkle, and cercospora leaf spot, which cause substantial productivity losses. Despite expanding applications of deep learning in plant disease diagnosis, systematic multi-architecture evaluation for mungbean disease classification under natural field conditions remains limited. This study addresses this gap by evaluating five state-of-the-art deep convolutional neural network (DCNN) architectures on a large-scale, field-acquired mungbean dataset that captures real-world variability across environmental conditions and disease severity levels, distinguishing it from controlled laboratory studies. A total of 5,617 original images across five classes were used. Data augmentation was applied exclusively to the training subset after stratified splitting to prevent data leakage. The dataset was partitioned into training (70%), validation (15%), and testing (15%) subsets. VGG16, VGG19, ResNet50V2, DenseNet121, and InceptionV3 were evaluated using identical transfer learning and fine-tuning protocols. Model performance was assessed using AUC-ROC, Cohen's kappa coefficient, McNemar's test for pairwise statistical comparisons, five-fold cross-validation, and Grad-CAM-based interpretability. On the independent test set, InceptionV3 achieved the highest accuracy (98.47%) and macro-F1 (98.49%), followed by VGG16 (98.36%) and VGG19 (97.89%). AUC-ROC values exceeded 0.997 for all models, confirming excellent class discrimination. Grad-CAM visualizations further confirmed that model predictions were based on biologically relevant disease symptoms. The findings demonstrate the effectiveness of deep learning for robust disease recognition under realistic field conditions and highlight the potential of AI-based diagnostic tools for crop health monitoring, precision agriculture, and decision-support systems in mungbean production.

Shail Bala, S. I. Harlapur, A. Kanade et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.