Evaluating Collective Intelligence: A Conceptual Framework for Benchmarking Single-Agent vs. Multi-Agent Systems in the Google A2A Protocol

Abstract

The Agent-to-Agent (A2A) Protocol was developed and released as an open-standard, vendor-agnostic, AIinteroperable agent solution through Google, then transitioned to the Linux Foundation in April 2025. It was the first open standard to be developed to support A2A interoperable agents. Its primary design tenet is the architectural claim that A2A Agent Networks will be able to perform at least equally as well or better than Single Agent Systems (SAS) on complex tasks. As the central theme behind the rapid adoption of the A2A Protocol among over 50 enterprise partners, there does not exist any structured framework to provide comparative analysis of SAS with A2A Multi-Agent Systems (MAS) using controlled conditions. The purpose of this paper is to create a structural framework that can compare and evaluate SAS vs. A2A MAS. The proposed framework has four components: a $2 \times 2$ Task Taxonomy that classifies tasks by context scope and sub-task dependency; a five-dimensional metric space (accuracy, latency, token efficiency, failure rate and cost per successful task); a four control evaluation process; and a formal trade-off model between cost and accuracy that introduces the Coordination Overhead Ratio (COR), Accuracy Lift (AL), Efficiency Gain Threshold (EGT), and Network Overhead Factor (NOF). One of the most important conceptual findings from this framework is that, if compute budgets are held equal, the performance advantage of MAS is highly task type dependent and decreases significantly for well scoped, single context tasks. In addition, the proposed framework identifies five confounding categories: Compute Budget Asymmetry, Task Type Dependency, Model Power Attribution, Evaluation Metric Scope, and Agent Card Discovery Fidelity - which systematically invalidate naive SASversus-MAS comparisons presented in existing literature; empirical validation remains future work.