This paper distill four design principles for scalable MAS architectures from a structured analysis of prior work: simplicity, elastic feedback, sequential workflows with optional loops, and summary-based communication, and evaluates four configurations of increasing complexity on a standardized benchmark of terminal-based system engineering tasks.
Abstract
LLM-based multi-agent systems have the potential to enable collective intelligence and scale toward solving highly complex tasks through coordinated ensembles of specialized agents. However, despite their theoretical potential, the architectural design space remains largely non-systematized and lacks broadly established design principles. Furthermore, the scalability characteristics of such systems are only partially understood so far. This paper makes two contributions. We first distill four design principles for scalable MAS architectures from a structured analysis of prior work: simplicity, elastic feedback, sequential workflows with optional loops, and summary-based communication. We operationalize these principles in a reference architecture whose topology is formalized as a constrained directed workflow graph, and we evaluate four configurations of increasing complexity on a standardized benchmark of terminal-based system engineering tasks using two LLMs of differing capability. Our findings show that scaling yields measurable accuracy improvements with approximately linear cost growth, but only when the underlying LLM exceeds a minimum capability threshold. Performance peaks at intermediate complexity, then degrades due to timeouts and evaluation limitations. In addition, persistent consistency issues emerge as a central challenge across all scaling levels. These results provide concrete design guidance for practitioners and highlight consistency and evaluation standardization as key targets for future research.
This survey presents a systematic taxonomy and technical review of dynamic orchestration strategies designed to address communication overhead, KV cache management challenges, and increased token consumption within large Language Model-based Multi-Agent Systems.
Heet Nagoriya, H. Raithatha· International Journal of Kno...· 0 citations
EASy is proposed, a trainable agentic framework that jointly optimizes task performance and computational efficiency through reinforcement learning and consistently achieves stronger performance-efficiency trade-offs than strong agentic baselines.
Junnan Liu, Linhao Luo, Thuy-Trang Vu et al.· 0 citations
Large Language Models (LLMs) have demonstrated remarkable capabilities as powerful components in agentic systems, enabling sophisticated reasoning and complex task execution. However, current approaches to manually designing and optimizing agentic systems heavily rely on manual effort, limiting their adaptability and scalability. Recent work has explored the automated optimization of workflow designs. However, these approaches often overlook the crucial role of model capabilities and focus on single performance metrics, failing to address real-world deployment constraints. In this paper, we present AgentFactory, a framework that jointly optimizes both foundation models and workflow structures in agentic systems while considering multiple objectives including performance, cost, and efficiency. AgentFactory leverages advanced LLMs as optimizers to navigate the vast search space of possible configurations, employing a three-stage optimization pipeline to automatically discover effective combinations of fine-tuned models and optimized workflows. Through an iterative optimization process, our framework systematically explores and evaluates different agentic system designs, adapting to task-specific requirements while maintaining operational efficiency. We evaluate AgentFactory across eight benchmarks spanning five domains, including general reasoning, coding, mathematics, medicine, and finance. Our experiments demonstrate that AgentFactory consistently outperforms both manually designed methods and existing automated approaches, achieving an average improvement of 9.1% across all benchmarks, with particularly significant gains in domain-specific tasks (19.6% on MedQA and 18.7% on FinEval). These results establish AgentFactory as a promising approach for developing more capable and efficient agentic systems through automated optimization.
En-Ci Zhang, Hao-Fen Wang, Yuesheng Zhu et al.· Pacific Rim International Co...· 0 citations
Agent-based models (ABMs) have long been employed to explore how individual behaviors aggregate into complex societal phenomena in urban space. The rise of Large Language Models (LLMs) offers a new paradigm for ABM, enabling high-fidelity modeling of human behavior. However, urban dynamics often require extensive agent interactions to emerge, and scaling up LLM agent simulations is limited by high-latency remote LLM inference and high costs. To address this, we propose the OpenCity framework for large-scale LLM agent simulation. We design a novel "Group-and-Distill" strategy that significantly reduces costs without sacrificing behavioral diversity by aggregating agents with shared static characteristics while preserving their unique dynamic states. Moreover, a system-level request scheduler is designed to optimize I/O concurrency and communication bottlenecks. Experiments in six cities globally demonstrate that OpenCity achieves a 600x speedup and a 45% reduction in token usage compared to standard baseline models, while maintaining 96% behavioral consistency with independently invoked agent baselines. Based on OpenCity, we conduct first benchmark test on large-scale social segregation with LLM agents. The results show that OpenCity-accelerated generative agents are effective in reproducing socioeconomic segregation patterns. Furthermore, our scalability analysis demonstrates that the realism of emerging social phenomena increases as the agents scale up, quantitatively validating the necessity of large-scale simulations. Finally, we demonstrate the framework's capability as a policy sandbox through a counterfactual analysis of urban equity. Overall, OpenCity provides a robust and efficient framework that unlocks the potential of LLMs for large-scale, high-fidelity multi-agent systems analysis. Code repo is available at https://github.com/tsinghua-fib-lab/OpenCity.
Qingbin Zeng, Yuwei Yan, Zhiheng Zheng et al.· Proceedings of the 32nd ACM...· 0 citations
This work presents OptiMAS, a task-agnostic agentic optimizer that leverages textual interaction trajectories and task feedback as loss signals for end-to-end MAS evolution and sustains performance improvement over extended optimization horizons.
Yuxin Cheng, Chang Liu, Hanxin Yu et al.· 0 citations
MANTA, a framework for Multi-Agent Network Topology Adaptation that enables communication structures to self-evolve at inference time, is introduced and shows that inference-time self-improvement can extend to the architecture of collaboration itself.
Mao-Xun Huang, Jerry Wang, Yi-Cheng Lai et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
