Comparative assessment of large language models for microbial phenotype assignment

Abstract

Large language models (LLMs) are increasingly used to extract knowledge from text, yet their coverage and reliability in biology remain unclear. Microbial phenotypes are especially important to assess, as comprehensive data remain sparse except for well-studied organisms and they underpin our understanding of microbial characteristics, functional roles, and applications. Here, we systematically assessed the biological knowledge encoded in publicly available LLMs for structured phenotype assignment of microbial species. We evaluated the performance of up to 57 LLMs across different experiments, including state-of-the-art models such as Claude Sonnet 4 and the GPT-5 family of models. Across phenotypes, LLMs reached accurate assignments for many species, but performance varied widely by model and trait, and no single model dominated. Model self-reported confidence is informative, with higher confidence aligning with higher accuracy, and can be used to prioritize phenotype assignment, effectively distinguishing between high- and low-confidence inferences. Overall, our study outlines the utility and limitations of text-based LLMs for phenotype characterization in microbiology.