Social simulations with large language model risk Utopian illusion

Abstract

Reliable simulation of human behaviour is essential for explaining, predicting, and intervening in our society. Recent advances in large language models (LLMs) have shown promise in emulating human behaviours, interactions, and decision-making, offering a powerful new lens for social science studies. However, the extent to which LLMs diverge from authentic human behaviour in social contexts remains underexplored, posing risks of misinterpretation in scientific studies and unintended consequences in real-world applications. This study introduces a systematic framework for analysing LLMs’ behaviour in social simulation. Our approach simulates multi-agent interactions through chatroom-style conversations and analyzes them across five linguistic dimensions, providing a simple yet effective method to examine emergent social cognitive biases. We conduct extensive experiments involving eight representative LLMs across three families. The results show that LLMs do not faithfully reproduce genuine human behaviour but instead reflect overly idealised versions of it, shaped by the social desirability bias. In particular, LLMs show social role bias, primacy effect, and positivity bias, resulting in “Utopian” societies that lack the complexity and variability of real human interactions. These findings suggest that LLM-based simulations reproduce how people wish others to behave rather than how people themselves behave, a divergence that amplifies through multi-agent interaction and, if uncritically adopted in social analysis, may produce an artificially narrowed representation of social life. This paper calls for more socially grounded LLMs that better capture the diversity of human social behaviour.