Hierarchical cross-modal spatiotemporal attention network for crop yield prediction in complex Himalayan agroecosystems

Abstract

Introduction Accurate crop yield prediction is essential for precision agriculture, climate-resilient farm management, and timely decision-making. However, yield prediction remains challenging because of complex, nonlinear, and multi-scale interactions among crop physiology, soil conditions, weather variability, and management practices. Conventional statistical, machine learning, and deep learning approaches, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and long short-term memory (LSTM) networks, often have limitations in capturing long-range temporal dependencies, hierarchical spatial relationships, heterogeneous data modalities, and predictive uncertainty. Methods This study proposes a Hierarchical Cross-Modal Spatio-Temporal Attention Network (HCM-STAN) for crop yield prediction using a unified attention-based framework for integrating heterogeneous agro-environmental data. The architecture incorporates four principal components: (i) hierarchical spatio-temporal attention operating across tree, block, orchard, and regional scales; (ii) a cross-modal attention module for adaptive fusion of IoT-based soil sensor observations, weather data, plant physiological measurements, and drone-based multispectral imagery; (iii) a soil-aware attention head that explicitly models soil memory, hysteresis effects, and root-zone dynamics; and (iv) an uncertainty-aware prediction module combining Monte Carlo dropout and quantile regression to generate calibrated prediction intervals. The framework was evaluated using a multi-source dataset collected from Himalayan apple orchards during 2023–2026, comprising 1,247 monitored trees across three commercial orchards. Results The proposed HCM-STAN achieved an R2 of 0.89 for four-week-ahead crop yield prediction, representing a 15.6% improvement over the CNN-LSTM baseline. Furthermore, zero-shot transfer to mango orchards achieved an R2 of 0.67, corresponding to approximately 75% of the performance obtained under full training. Discussion The results demonstrate that integrating hierarchical spatio-temporal representation learning, cross-modal attention, soil-aware modelling, and uncertainty estimation can effectively address the heterogeneous and dynamic nature of crop yield prediction in complex mountain agroecosystems. The observed zero-shot transfer performance further indicates the potential of HCM-STAN to generalize across crop systems without crop-specific retraining. Overall, the proposed framework provides a promising foundation for reliable, uncertainty-aware yield forecasting and can support precision agriculture, climate-smart farm management, and data-driven decision-making in heterogeneous agroecosystems.