Performance within the foundation paradox: the case for small qualitative evaluations

Abstract

Large language models (LLMs) face an evaluation and performance paradox. As they grow more capable, models are increasingly deployed as sources of guidance and advice across diverse domains. Yet prevailing evaluation practices remain concentrated on either well-bounded, proceduralized tasks validated through large-scale benchmarks or highly specialized expert queries that are shaped by formal professional standards. Between these regimes lies a growing class of intermediate, situational, and interdisciplinary uses that are neither reducible to fixed answers nor stabilized by consensus authority, but that increasingly shape real-world AI deployment. This paper examines this evaluative gap and introduces small qualitative evaluations (SQEs) as a human-in-the-loop framework for assessing LLM performance in such less-bounded domains. Drawing on principles from qualitative data analysis, SQEs emphasize analytic parsimony, theory-grounded rubric construction, and structured human judgment. We illustrate this approach through the Desert Language Model Evaluation Framework (DLEF), developed within resilience and sustainability research to evaluate open-ended guidance in the context of extreme heat adaptation. Using GPT-4o as a demonstration case, we show how SQEs surface systematic differences in feasibility, place-based reasoning, and executive clarity, with inter-rater reliability strongest for adaptation-relevant dimensions (kappa = 0.56 to 0.91). To test scalability, we conduct a systematic replication study in which four open-weight models were used to automatically score study responses across eight prompt variants, finding that automated replication success correlates significantly with human inter-rater reliability across dimensions (Pearson r = 0.686, p = 0.005). Linear probe analysis of last-layer hidden states further reveals that this pattern is reflected in model representations: dimensions where human coders agree are geometrically organized in ways consistent with the human construct, while dimensions that resist human agreement are either not reliably encoded or are encoded in ways that reflect the model’s own interpretation of the construct rather than the rubric’s intent. These findings suggest that human inter-rater reliability functions not only as a quality check on the rubric, but also as a prior estimate of automated replication feasibility. We argue for SQEs as a complementary evaluation strategy for AI systems operating in dynamic, contested, and interdisciplinary settings.