Confidence Adaptive Agent Negotiation (CAAN): An Evidence-Grounded Consensus Framework for Multi-Agent Large Language Model Systems
Abstract
Multi-agent LLM systems are a natural fit for complex reasoning tasks, yet most existing frameworks handle conflicting agent outputs through majority voting or confidence-based selection, without ever checking the evidence behind those outputs. We propose Confidence Adaptive Agent Negotiation (CAAN), an evidence-grounded consensus framework in which specialized agents solve a problem independently, gather supporting evidence, and justify their reasoning rather than simply report how confident they are. At the center of CAAN is a Reliability Evaluation Module (REM), which scores each agent on evidence relevance, source credibility, requirement coverage, logical consistency, and constraint satisfaction to produce a dynamic reliability score. During negotiation, agents exchange verified evidence and revise their positions based on these objective evaluations rather than on subjective agreement, and the final decision is reached through trust-weighted consensus that combines individual reliability with collective agreement. In a 60-task pilot study, this approach reduced unsupported conclusions and produced more explainable decisions than conventional confidence- or voting-based methods, suggesting the framework generalizes across evidence-driven reasoning tasks - though we treat this as preliminary evidence rather than a conclusive claim, pending large-scale evaluation.