A TBC-TBA self-refine learning multi-agent framework that enables dynamic normative adaptation through iterative multi-agent feedback that integrates Think-Before-Chat (social feedback processing) and Think-Before-Act (norm-guided decision making) phases, allowing agents to progressively refine their normative understanding via structured interaction cycles.
Large language models (LLMs) now power the reasoning core of intelligent virtual agents deployed across an expanding range of social settings, from tutoring students and supporting patients in healthcare, to mediating group discussions and representing humans in various social settings. Effective deployment demands social cognition, the capacity to model what others believe, detect deception, and coordinate strategic action under incomplete information. These capacities, exemplified in the social dynamics of the game Among Us, remain poorly characterized in current LLM evaluation frameworks. We introduce a strategic game arena that situates LLM agents in social deduction scenarios inspired by Among Us, requiring theory of mind, deception detection, and cooperative deliberation under uncertainty. We evaluate 19 open-weight models across 10,134 games and 289,614 utterances, testing both homogeneous and heterogeneous crews. Our experiments reveal three findings. First, crewmates voting through generative reasoning reach only \(50.4\% \pm 4.4\%\) F1 when identifying imposters, while a logistic regression classifier trained on the same discussion transcripts achieves \(85.3\%\) F1. Second, scaling model parameters yields a statistically significant but practically marginal improvement. Medium models (60–82B) reach \(52.8\%\) F1 against \(46.1\%\) for small models (7–20B), a 6.7 point gain (Mann–Whitney U, p = 1.5 × 10− 34). Third, agents fail to integrate evidence coherently during deliberation. Imposters self-incriminate in \(4.12\%\) of their statements, yet crewmates eject the confessing agent only \(33.8\%\) of the time. Crewmates reverse their stated suspect between consecutive rounds without new justification in \(41.9\%\) of cases. Sentiment remains uniformly neutral whether an agent is reporting a body or delivering a routine update. These gaps identify concrete limits in the social cognition of current LLM-powered agents and motivate architectural changes for virtual agents that must cooperate with humans. The source code and the live arena viewer are available at https://ufdatastudio.com/projects/agents-among-us.
Kevin Kurian, Kevin Scroggins, Emmanuel Dorley et al.· Proceedings of the 26th ACM...· 0 citations
Results show that TRACE converts high model potential into stable, consistent performance gain, and bridge the gap between potential and reliable performance to just 4.0 points.
Wenhao Wu, Menghao Zhang, X. Wang et al.· 0 citations
With increasingly powerful large language models (LLMs) and LLM-based agents tackling an ever-growing list of tasks, we envision a future where numerous LLM agents work seamlessly with other AI agents and humans to solve complex problems and enhance daily life. To achieve these goals, LLM agents must develop collaborative skills such as effective persuasion, assertion and disagreement, which are often overlooked in the prevalent single-turn training and evaluation of LLMs. In this work, we present Collaborative Reasoner ( Coral ), a framework to evaluate and improve the collaborative reasoning abilities of language models. In particular, tasks and metrics in Coral necessitate agents to disagree with incorrect solutions, convince their partners of a correct solution, and ultimately agree as a team to commit to a final solution, all through a natural multi-turn conversation. Through comprehensive evaluation on six collaborative reasoning tasks covering domains of coding, math, scientific QA and social reasoning, we show that current models cannot effectively collaborate due to undesirable social behaviors, collapsing even on problems that they can solve singlehandedly. To improve the collaborative reasoning capabilities of LLMs, we propose a self-play method to generate synthetic multi-turn preference data and further train the language models to be better collaborators. Experiments with Llama-3.1 , Ministral and Qwen-2.5 models show that our proposed self-improvement approach consistently outperforms finetuned chain-of-thought performance of the same base model, yielding gains up to 16.7% absolute. Human evaluations show that the models exhibit more effective disagreement and produce more natural conversations after training on our synthetic interaction data. 1
Ansong Ni, Ruta Desai, Yang Li et al.· Neural Information Processin...· 7 citations
Large language model-based multi-agent systems (LLM-MAS) are designed through roles, debate protocols, and aggregation rules. These choices create implicit social expectations: agents may be expected to trust, challenge, defer to, or collaborate with peers. We study the effects of making inter-agent relation semantics explicit. We use a minimal signed-network formulation of relational priors and inject natural-language renderings into agent system prompts while holding the task protocol fixed. Across a commons-governance simulation and multi-agent debate, relational priors primarily act as convergence pressure: increasing relational positivity tends to make agents coordinate or agree more readily. This pressure can help when utility rewards behavioral alignment, as in sustainable resource governance and subjective consensus. It does not, however, reliably improve accuracy. In objective QA debates, higher positivity can increase agreement even when correctness-conditioned agreement does not improve and may decline in some settings. Effects vary by model backbone, relation type, and topology; explicit neutrality is not equivalent to omitting relational framing. We argue that relational priors should not be a default add-on for LLM-MAS. Their safer use is diagnostic and task-specific: compare against a no-prior baseline, monitor correctness-conditioned metrics when truth matters, and omit the relational layer when validation does not justify it.
Ming Shen, Chao Shang, Sadat Shahriar et al.· 0 citations
Knowledge Distillation is a widely adopted technique in the training and fine-tuning of large language models (LLMs) enabling transfer of structured information and functional behavior from a large teacher model to a smaller student model while significantly reducing computational costs. However, as the use of distillation increases in both scale and complexity it raises an important question about what kind of knowledge is really transferred from the teacher model. In this work, we argue that apart from the functional knowledge, student models also learn behavioral patterns, specifically how a model represents its own identity raising concerns about output homogeneity, model biases, and accountability. To address this challenge, we introduce STEMMA, a multi-modal and multi-agent framework in which role specific agents collaboratively probe self identification behavior in different models. We also contribute a set of adversarial prompts designed manually to evaluate identity consistency in LLMs. Our results show that to an extent most models are vulnerable to inconsistencies in self-representations.
This work proposes a novel neuro-symbolic fast-slow thinking (NeSyFS) framework for LLM agent, addressing the challenges introduced by partial observability in a unified approach, and uses a knowledge graph to represent the belief state.
Duo Xu, F. Fekri· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
