It is suggested that multi-agent debate is most valuable when it preserves independent informational signals rather than simply improving measured reasoning quality, and that interventions enforcing stronger causal reasoning do not improve financial performance.
Abstract
Large language models (LLMs) are increasingly used for financial decision-making, yet it remains unclear whether improvements in reasoning quality translate into better economic outcomes. We investigate this question using a multi-agent debate framework for portfolio allocation in historical market simulations, where specialized agents propose, critique, and revise investment decisions. Reasoning quality is evaluated across four dimensions: logical validity, evidential support, alternative consideration, and causal alignment, and compared with downstream financial performance. Across 210 controlled runs, aggregate reasoning quality shows no meaningful relationship with Sharpe ratio (r = 0.07, p = 0.29) or total return (r = 0.03, p = 0.70). Structured prompting increases measured reasoning quality from about 0.72 to 0.84 (+17.7%, Cohen's d about 2.0), but these gains do not consistently translate into higher returns. We identify sycophantic convergence as a central failure mode, where agents abandon independent positions during critique-revision cycles and converge toward similar allocations. A Jensen-Shannon divergence intervention that preserves disagreement improves Sharpe by +0.14 (p = 0.028) and Sortino by +0.25 (p = 0.026), while interventions enforcing stronger causal reasoning do not improve financial performance. Our results suggest that multi-agent debate is most valuable when it preserves independent informational signals rather than simply improving measured reasoning quality.
It is shown that additional reasoning can change financial decisions without reliably improving their economic value, motivating validation for each task before deployment.
Large language models (LLMs) are being deployed at scale in consequential real-world systems, from financial markets to content moderation to hiring. We show that improving individual model capability can degrade rather than improve system-level outcomes. We hypothesize that shared training and architectures can lead m...
Jillian Ross, Eric C. So, Zoe De Simone et al.· 1 citation
Large language models (LLMs) and multi-agent systems (MAS) have shown promise in financial decision-making, yet existing evaluations focus on equity trading and primarily assess directional prediction, overlooking the structural complexity of derivative markets. Option trading introduces fundamentally different challen...
Hao-Chen Luo, Yi-Fan Li, Binh Minh An et al.· 0 citations
Controlled evidence ablations show that recent trading history strongly governs activity prediction, whereas asset selection is substantially more sensitive to the available evidence.
Multi-agent reasoning with large language models (LLMs) raises a central question for collective decision-making: when agents disagree, whose decision should be trusted? Resolving such disagreements is difficult because agents'decisions are susceptible to non-semantic biases such as majority pressure and response-style...
Sungjae Lee, Soheil Abbasloo, Jabin Koo et al.· 0 citations
Large language model (LLM)-based financial agents use market information to generate trading actions together with textual reasoning. However, the resulting executable position transition may conflict with its supporting analysis, while the effect of intervening on that transition becomes observable only after the mark...