CARTE: An LLM-based automated tool for semantic clustering and switching identification in associative verbal fluency

Abstract

Semantic Verbal fluency is a core component of neuropsychological assessment,valued for its clinical sensitivity and ease of administration. Yet, its full potential is rarelyfully exploited: performance is typically reduced to a total word count, overlooking the richdynamics of semantic foraging, such as clustering (exploitation within a semantic category)and switching (exploration across categories). Although these dynamics are central to modelsof memory search, they require laborious manual coding, limiting their large-scale analysisand use for scoring. Automated alternatives exist, but none can identify which specificcategory each response belongs to, and their agreement with manual scoring remains onlymoderate.Here, we introduce CARTE (Category Attribution and Response TransitionsEvaluation), an automated scoring method that identifies semantic categories and detectsclustering/switching transitions in responses to the Polysemous Fluency Task (PolyFT).Because PolyFT is an associative verbal fluency task relying on polysemous cue-words,semantic categories map naturally onto distinct cue-meanings, making them easier to identify- a process we automate in CARTE by means of a large language model. Analyzing responsesfrom 90 French-speaking participants across 35 polysemous cues, we show that CARTEoutperforms existing automated methods in matching human raters. Crucially, CARTE’sperformance is independent of sample size: once classified, a word is permanently indexed ina cumulative semantic database, ensuring computational efficiency, reproducibility, andscalability across studies. By bridging cognitive richness and methodological feasibility, thiswork offers a simple yet robust tool for studying the dynamics of semantic memory search.