The present study profiled eight Serbian participants through a questionnaire, a deep interview, and a written self-presentation, recorded their reactions to sixty-eight social media posts, and asked four language models to predict those reactions under five prompt conditions varying profile content and instruction style.
Abstract
Platform policies are increasingly tested on artificial users, making agent fidelity important. Yet convincing fake profiles could also manipulate perceived public opinion before elections. Validation has concentrated on agreement with human behaviour and has paid little attention to whether an agent behaves in line with the profile it was given. The present study profiled eight Serbian participants through a questionnaire, a deep interview, and a written self-presentation, recorded their reactions to sixty-eight social media posts, and asked four language models to predict those reactions under five prompt conditions varying profile content and instruction style. Attitudinal content improved prediction over demographic backstories by a wide margin. Agents matched their stated profiles more closely than participants matched their own survey answers, and consistency proved unrelated to fidelity once profile information was present. Instructing models to respond intuitively and immediately rather than analytically gave the highest fidelity of any condition and cut the compression of individual differences from seven times the human level to three. The advantage held on posts about topics the questionnaire never raised, where that condition reached the highest fidelity of any setup and beat a crowd baseline by a wide margin, which suggests that agents prompted this way could serve as general-purpose simulated users rather than specialists on the topics they were profiled for. Results may bear implications for the development of language models, because intuition-based setups appear better suited to some tasks than reasoning-based ones.
A dual-level evaluation framework to assess LLM-based agents at both the individual and collective levels is proposed, finding that while agents capture broad partisan orientations, they underestimate within-group variability and reproduce stereotypical ideological biases.
M. Al Ali, Filip Mihai Muntean, Lucia Donatelli et al.· International Conference on...· 1 citation
Social media platforms mediate how billions form opinions and engage with public discourse. As autonomous AI agents increasingly participate in these spaces, understanding their behavioral fidelity becomes critical for platform governance and democratic resilience. Previous work demonstrates that LLM-powered agents can...
Ljubiša Bojić, Alexander Felfernig, Bojana M. Dinić et al.· Scientific Reports· 1 citation
Understanding the dynamics of stance change on social media is crucial for addressing polarization and information integrity, yet observational studies face challenges including limited experimental control, restricted data access, and algorithmic confounds. We leverage Generative Agent-Based Modeling (GABM)—a novel si...
Valerio La Gatta, Gian Marco Orlando, Marco Perillo et al.· Proceedings of the 37th ACM...· 1 citation
Abstract Conversational AI has been proposed as a scalable way to correct public misconceptions, yet its effectiveness may depend on perceptions of political neutrality. As large language models become objects of partisan conflict, political elites increasingly portray them as ideologically aligned. We test whether suc...
M. DiGiuseppe, J. Robison· PNAS Nexus· 0 citations
This Work-in-Progress examines whether personality-informed prompting changes perceived LLM emotional support in driving scenarios designed to elicit stress. A condition-order-balanced, within-subject CARLA simulator study (n = 14) compared a baseline with a Driver Personality Profile (DPP) condition; a supplementary o...
Max Mittelstädt, Ece Sutanrikulu, Lumbardh Ljatifi et al.· International Conference on...· 0 citations
Generative agent-based models (GABMs) are increasingly used to simulate social media dynamics, including misinformation spread. For such social simulations to be valid proxies of human behavior, LLM agents should replicate established human cognitive biases, among them the Illusory Truth Effect (ITE), where repeated ex...
A new method, called CW-Net, translates the reasoning process of an autonomous vehicle’s AI system into understandable concepts that explain its behavior.