Skip to content
Preprint

A game theory for foundation models shows new paths to rational cooperation through similarity inference

Aug 2026 · 1 citation
Computer Science

TL;DR

A striking finding is reported: when interacting in stylized social dilemmas, foundation model agents engaging in optimal planning consistently converge to stable cooperation, directly contradicting classical game-theoretic predictions of mutual defection.

Abstract

As autonomous agents powered by foundation models are increasingly integrated into social and economic systems, understanding the principles governing their collective behavior is essential for ensuring safety and cooperation. Classical game theory, the dominant framework for modeling rational interaction, is built upon the assumption of `decoupled agency,'where agents treat their own decision-making as independent of the environment and other actors. Modern AI agents, however, jointly predict their own future actions alongside external observations. Here, we report a striking finding: when interacting in stylized social dilemmas, foundation model agents engaging in optimal planning consistently converge to stable cooperation, directly contradicting classical game-theoretic predictions of mutual defection. To understand this phenomenon, we introduce the `embedded Bayesian agent,'a theoretical model for foundation model agents. By shifting from decoupled to embedded agency, these agents model themselves as part of the universe they inhabit, maintaining epistemic uncertainty about their own decision-making algorithms. We show that by inferring whether others are behaviorally similar, an embedded agent treats its own deliberation during planning as evidence: a decision to cooperate predicts a similar decision by a similar partner. We formalize this mechanism of similarity inference through the `embedded equilibrium,'a novel solution concept replacing the Nash equilibrium to provide a foundational game theory for the social behavior of modern AI agents.

View source

Similar papers

Open access Sep 2026

Research on the Game Dynamics of Optional Public Goods

The main findings demonstrate that individuals update strategies primarily through self-adjustment based on historical payoffs, with imitation playing merely an auxiliary role and the optimal self-adjustment proportion is approximately 0.9, and low sensitivity coefficients and mutation rates favor the emergence of coop...

Hao-Chen Wu, Meng-Cheng Sun, Lu-He Yang et al. · 0 citations

SAGE: Structured Strategic Reasoning for Efficient LLM Game Playing

SAGE is a training-free inference-time framework that structures LLM strategic reasoning around three coordinated operations: anchor, adapt, and recalibrate, and distills strategically related interactions into counterfactual hypotheses about previously missing considerations, allowing past experience to recalibrate th...

Zhi-Wei Chen, Tian-Chun Wang, Zhong-Tao Rao et al. · 0 citations
Sep 2026

Agent-based Modeling: Equilibrium, Echo Chambers, and Efficiency in Hybrid Coevolutionary Opinion Games

The Hybrid Coevolutionary Opinion Game (H-COG) is introduced, in which analytical and LLM-driven agents share one network, choose their neighbors by opinion similarity in every round, and hold stances drawn from real Reddit comments on gun control and abortion.

Ming Jiang, An-Tzu Teng, Jun-En Liu et al. · 1 citation
#artificial intelligence Preprint Oct 2026

Identity, Incentives, and Cooperative Behavior for AI: Reverse-Engineering Multiagent Cooperation

This work develops a framework for reverse-engineering why a given set of agents engage in a particular set of cooperation-related behaviors and argues that many different mechanisms can often produce similar observed behaviors and introduces a set of experimental designs along with conditions under which they can succ...

A. Perrault, Bryan Wilder · 0 citations
#artificial intelligence Open access Sep 2026

Recursive Reasoning or Statistical Extrapolation? In-Context Learning in Multi-Agent Interdependent Decision-Making

This work extends the mechanistic study of ICL to strategic multi-agent settings, introduces REE as a diagnostic tool for distinguishing reasoning from extrapolation, and provides a reusable framework for probing the boundaries of LLM reasoning in recursive belief tasks.

Y. Liu, Wen-Wen Li, Yi-Fan Dou et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.