Investigating Cross-Modal Semantics in Large Language Models Using Concept-Color Associations
Cross-modal correspondences, exemplified by systematic mappings between abstract concepts and colors, illuminate the mechanisms through which humans integrate sensory inputs with conceptual frameworks. Existing theories propose that these associations arise from semantic mediation via metaphors, statistical regularities in language and environment, or overlapping affective or structural features. Trained solely on textual data, large language models provide a unique perspective for examining whether such mappings can emerge from linguistic patterns alone. In this study, we collected color associations for 85 abstract concepts across temporal, alphanumeric, directional, and spatial categories from 260 Chinese university students. These human responses were then compared with outputs from 10 LLM variants across three families, GPT, Deepseek, Doubao, generating 300 responses per concept per model to capture distributional tendencies. Humans exhibited consistent conceptcolor links for 72 of the 85 concepts, with semantically proximate items, such as consecutive seasons or months revealing organized similarities in their color profiles. LLMs, in contrast, produced sharper and more peaked distributions, aligning well with humans on established conventions such as spring-green or A-red, yet diverging on others, for instance associating the concept 0 predominantly with black rather than the human preference for white. Alignment metrics were moderate overall, with the highest levels observed in the GPT family, suggesting that expansive training corpora better approximate nuanced human variability.