Optimizing large language model prompts for biomedical knowledge discovery

Abstract

Biomedical knowledge extraction is fundamental to understanding complex biological systems, yet they remain challenging due to the rapid growth of biomedical literature, the heterogeneity of data sources, and the limitations of traditional rule-based and supervised learning approaches. Large language models (LLMs) have shown considerable promise for biomedical knowledge retrieval and discovery by demonstrating strong capacities for contextual reasoning and information extraction; however, their reliability, robustness, and interpretability in biomedical applications remain insufficiently understood. In particular, systematic evaluations against curated biological ground truth, multimodal pathway analysis, and scalable prompt optimization have received limited attention. This work presents a comprehensive investigation of LLMs for biomedical knowledge extraction, with a focus on gene–gene regulatory relation prediction, pathway component recognition, multimodal pathway figure understanding, and automated prompt optimization. First, a large-scale benchmarking study evaluates multiple proprietary and open-source LLMs using curated ground-truth data from the Kyoto Encyclopedia of Genes and Genomes (KEGG), which reveals substantial variability across models and persistent challenges in recall, directional inference, and hallucination control. Second, this work extends biomedical knowledge extraction beyond text by introducing a Genetic Prompt Vision framework for multimodal pathway image analysis. This framework employs evolutionary optimization to automatically generate effective multimodal prompts for extracting gene–gene interactions from biological pathway figures. Experimental results demonstrate that vision–language models enhanced through automated prompt optimization outperform traditional OCR-based pipelines in extraction accuracy and biological interpretability. Third, this study proposes a Bayesian Optimization Prompt framework for automated prompt optimization in biomedical relation extraction. This model-agnostic approach identifies robust prompts of ASCII strings without accessing model internals or performing fine-tuning. Across multiple LLMs, automated prompt optimization improves predictive performance while enhancing internal representation quality, as evidenced by improved semantic clustering and representation stability. Overall, this work establishes a scalable, reproducible framework for evaluating, optimizing, and interpreting LLMs for biomedical knowledge extraction. The findings provide practical insights into the strengths and limitations of LLMs in biomedical contexts and contribute methodological advances toward a reliable, interpretable, and multimodal LLM-based bioinformatics system.