Automating Content Analysis With Multiple LLM Agents: Impacts of Agent Attributes and Human–AI Collaboration

P ��������� : Choosing Your Experts Enhances Critical Thinking in Multi-Agent Research Ideation

Identifying Interaction Dimensions and their Relation to Agent Behavior in Human-AI Teams

Identifying and promoting effective user interaction strategies in human-AI interaction is critical to improving collaboration quality. However, what motivates users or how agent behavior affects them remains unclear. We analyzed self-reported user strategies in a study (N=60) using a collaborative game with three AI partners reflecting leader, follower, and shifting initiative behavior. We identified four interaction dimensions — intention, perception, communication, and coordination — and assessed their relation to agent behavior, team performance, and user preferences. We observed that while users self-report goal prioritization independently of agent behavior, in-game logs reveal that non-collaborative goals are prioritized significantly more often with AI leaders than with followers. Moreover, users tend to construct plans early through observation when the AI leads, yet they are more likely to use strategies to propose goals when the AI follows. Our work provides a new framework for AI agent designers to understand user behavior and guidelines for better supporting it.

Collaborative Reasoner: Self-Improving Social Agents with Synthetic Conversations

With increasingly powerful large language models (LLMs) and LLM-based agents tackling an ever-growing list of tasks, we envision a future where numerous LLM agents work seamlessly with other AI agents and humans to solve complex problems and enhance daily life. To achieve these goals, LLM agents must develop collaborative skills such as effective persuasion, assertion and disagreement, which are often overlooked in the prevalent single-turn training and evaluation of LLMs. In this work, we present Collaborative Reasoner ( Coral ), a framework to evaluate and improve the collaborative reasoning abilities of language models. In particular, tasks and metrics in Coral necessitate agents to disagree with incorrect solutions, convince their partners of a correct solution, and ultimately agree as a team to commit to a final solution, all through a natural multi-turn conversation. Through comprehensive evaluation on six collaborative reasoning tasks covering domains of coding, math, scientific QA and social reasoning, we show that current models cannot effectively collaborate due to undesirable social behaviors, collapsing even on problems that they can solve singlehandedly. To improve the collaborative reasoning capabilities of LLMs, we propose a self-play method to generate synthetic multi-turn preference data and further train the language models to be better collaborators. Experiments with Llama-3.1 , Ministral and Qwen-2.5 models show that our proposed self-improvement approach consistently outperforms finetuned chain-of-thought performance of the same base model, yielding gains up to 16.7% absolute. Human evaluations show that the models exhibit more effective disagreement and produce more natural conversations after training on our synthetic interaction data. 1

Agent‐Related Perceptions of Automated Versus Human Agents and the Role of Task‐Related Risk: A Meta‐Analysis

Firms often seek to replace human agents (HAs) with automated agents (AAs) for customer service, but customers still tend to react more negatively to AAs than to HAs. This meta‐analysis examines agent‐related perceptions (i.e., humanlikeness, warmth, and competence), which represent the key upstream customer reactions shaping downstream responses. Specifically, it clarifies the hierarchy of these perceptions and examines the moderating effect of task‐related risks (i.e., functional, psychological, and social) on these perceptions. Meta‐analytic path modeling with subgroup analyses, including 1191 effect sizes, provides novel insights. First, there is indication that perceived humanlikeness is not a prerequisite for perceived warmth and competence; all three perceptions emerge in parallel. Second, tasks entailing functional or psychological risk largely weaken the negative perceptions of AAs (vs. HAs). For psychologically risky tasks, AAs are not perceived as less competent than HAs. Tasks entailing social risk largely strengthen the negative perceptions of AAs (vs. HAs). These findings challenge two prevailing assumptions in the services marketing literature: that customers favor humans over AAs simply because they are not humanlike and that risk generally discourages AA usage. The results have important managerial and future research implications.

Principal Trait Analysis: Towards Deriving"Skills"in Human-AI Collaboration

An automated, data-driven approach to uncover patterns, which the authors term traits, of effective human-AI interaction that are aligned with task outcomes is explored and Principal Trait Analysis is proposed, a Principal Component Analysis-inspired algorithm for deriving common traits from patterns in LLM conversations.

AgentPanel: Toward a New Paradigm for Human--AI Collaboration in Exploring Scientific Questions

This work presents AgentPanel, a multi-agent forum for human--AI collaboration in scientific exploration, a multi-agent forum for human--AI collaboration in scientific exploration that outperforms a centralized multi-agent debate baseline and shows that users value AgentPanel for perspective diversity and exploration support.