BLAP: lesion-aware adaptive multi-scale visual prompt tuning for few-shot crop disease diagnosis
TL;DR
BLAP provides a lightweight solution that balances accuracy, efficiency, and interpretability for crop disease diagnosis in resource-constrained settings and may be extended to parameter-efficient fine-tuning of visionlanguage models in other domain-specific applications.
Abstract
Introduction Applying general-purpose vision-language models (VLMs) to crop disease diagnosis presents three critical bottlenecks: reliance on large-scale annotated data, the high computational cost of full finetuning, and existing adaptation methods designed mainly for discriminative classification without sufficient visual-linguistic interaction for generative diagnosis. Methods We propose BLAP, an adaptive multi-scale visual prompt fine-tuning framework built upon BLIP-2. BLAP introduces an adaptive visual prompt fusion module (APFM) with learnable prompt vectors and a gating mechanism, together with a multi-scale pyramid feature fusion module (PFM). All BLIP-2 backbone parameters are frozen, and only 0.11% of the model parameters are optimized. Results On a few-shot dataset comprising 990 images from 11 crops and 33 disease categories, BLAP achieved 92.78% recognition accuracy, outperforming the BLIP-2+LoRA baseline by 21.67 percentage points. BLEU-4 and ROUGE-L scores reached 0.6507 and 0.7184, respectively, while inference latency increased by only 2.15%. Discussion BLAP provides a lightweight solution that balances accuracy, efficiency, and interpretability for crop disease diagnosis in resource-constrained settings. The proposed dynamic prompt fusion and multiscale pyramid adaptation strategy may also be extended to parameter-efficient fine-tuning of visionlanguage models in other domain-specific applications.