The Systems Architecture of LLM Multi-Agent Systems: Routing, Memory, and Resource Optimisation

Abstract

Large Language Model (LLM)-based Multi-Agent Systems (MAS) have attracted significant research attention due to their ability to coordinate specialised agents for complex reasoning tasks. Their deployment within static, sequential multi-agent pipelines introduces communication overhead, KV cache management challenges, and increased token consumption. Recent empirical studies report that iterative refinement stages can account for up to 59.4% of total token consumption in software engineering workflows, with an average 2:1 input-to-output token ratio highlighting the impact of communication overhead in conventional architectures. This survey presents a systematic taxonomy and technical review of dynamic orchestration strategies designed to address these constraints. We examine routing and orchestration approaches including Directed Acyclic Graph (DAG)-based frameworks, semantic routing methods, graph-based optimisation techniques, and search-driven workflow planners, analysing their decision logic, computational overhead, and infrastructure trade-offs. We further review distributed state management techniques—including structurally lossless context trimming, tiered agentic memory systems, and test-time error pruning—that aim to mitigate state accumulation and reduce error propagation. A dedicated analysis of cost-aware routing via linear programming highlights the potential benefits of difficulty-aware model dispatch. Finally, we discuss hardware-aware routing and automated failure attribution as promising directions for future research. The surveyed literature highlights the growing importance of orchestration strategies, routing mechanisms, resource management techniques, and system-level evaluation methodologies for scalable deployment.