This work introduces Steiner's taxonomy of group tasks as a framework for analyzing multi-agent LLM scaling and shows that task structure is a fundamental determinant of team scaling.
Abstract
Multi-agent LLM systems are often expected to improve as team size increases, yet the scaling behavior may depend on task structure. Our central contribution is to introduce Steiner's taxonomy of group tasks as a framework for analyzing multi-agent LLM scaling and focusing the analysis on disjunctive and compensatory tasks. We model independently sampled agents as conditionally independent given the item, which yields their large-team limits: plurality voting converges to the model's modal answer, and averaging converges to the model's item-level bias. Across selected representative benchmarks, 13 open-weight models, and teams of up to 30 agents, we find qualitatively different scaling behavior. On disjunctive tasks, the probability that at least one agent is correct grows by 5-20 points with team size, but plurality voting over agents that answer directly realises almost none of this potential, as the model predicts to within 0.5 points on average. Multi-round revision raises accuracy considerably, yet the gain is nearly the same with one peer as with 29. In contrast, scaling provides little benefit on Fermi estimation, despite its natural suitability for aggregation: item-level biases shared across the samples of a model account for about 87% of the squared error, so averaging reduces error by only about 6%. Combining model families helps on Fermi estimation but does not surpass the strongest member on disjunctive tasks. These results show that task structure, together with the mechanism combining member outputs, is a fundamental determinant of team scaling.
The collaboration tax is formulated as the team-decentralisation loss of a two-player cooperative game with private information, with two propositions characterising its sign and its equivalence to a max-superadditivity violation.
Wei-Xiang Sun, Ze-Hong Wang, Hong Huang et al.· 2 citations
It is argued that when agents are aware of each other's model family, the group splits into clusters, where agents prefer interacting with others carrying their same label, although nothing in the task rewards or asks for such a split.
Xavier Del Giudice, Alessio Palma, Matteo Migliarini et al.· 0 citations
Agent evaluations should be evaluated as configurable systems themselves, and that some desired behaviors are more effectively implemented in the system than requested through prompting, according to a new benchmark of four scientific tasks.
Luis Wiedmann, Leander Girrbach, Cordelia Schmid et al.· 0 citations
The hypothesis that cognitive diversity among agents is the driver of multi-agent debate gains is tested, in the setting where the question is still measurable: small open-weight models with benchmark headroom.
The results suggest that multi-agent performance depends less on the number of agents than on the information they exchange, and interaction helps only when agents share the right information at the right time.
Summer Eunhyung Ann, Haokun Liu, Chen-Hao Tan· 5 citations
Change-of-measure lower bounds show that shared-reward identification is optimal up to one universal logarithmic factor, and that the entire statistical price of removing communication is a multiplicative $\rho^2$ in sample complexity.
Larissa Xu, Jasmine Nguyen, William Chang· 0 citations