Transformer-Based Hyperspectral Feature Learning Model for Early Detection of Wheat Yellow Rust

Abstract

Wheat yellow rust is a major threat to global food security, causing yield losses of up to 70% if not detected early. Hyperspectral imaging enables pre-symptomatic detection, but extracting discriminative spectral–spatial features is challenging due to high dimensionality, redundancy, limited labelled data, and subtle disease signatures. This study proposes a unified transformer-based framework that integrates self-supervised masked autoencoder (MAE) pretraining with a SpectralFormer classifier. The MAE learns spectral–spatial representations by reconstructing masked spectral tokens, thereby providing effective initialisation for downstream supervised learning. The pretrained encoder is subsequently fine-tuned for disease classification, and the fully integrated model is evaluated on a real-world UAV-acquired hyperspectral dataset. Results show that the proposed MAE-SpectralFormer achieves 98.6% accuracy, 98.4% F1-score, and 0.957 Receiver Operating Characteristic - Area Under the Curve (ROC-AUC), outperforming the strongest Convolutional Neural Network (CNN) baseline (Inception-ResNet) by 5.4 percentage points in overall accuracy and 6.0 points in Rust-class F1-score, and exceeding the supervised SpectralFormer by 2.6 and 2.4 points, respectively. These findings demonstrate that self-supervised spectral–spatial learning improves early disease detection and offers a scalable, data-efficient solution for hyperspectral crop monitoring.