Toward a Unified Security Systems Theory for Autonomous AI Systems

AIOracles: Modeling Agentic AI Security *

Governance and Security-by-Design: Embedding Safety and Alignment into Agentic AI Systems

As artificial intelligence systems transition from task-specific tools to autonomous agents capable of complex decision-making, traditional external oversight mechanisms become inadequate for ensuring safety, security, and alignment. This paper introduces a governance and security-by-design framework that embeds responsibility mechanisms directly into agentic AI architectures, enabling continuous self-monitoring and alignment verification through multi-agent governance systems. We demonstrate that external governance approaches fail to scale with system autonomy, creating temporal gaps between assessment and deployment that enable ungovernable behaviors. Through mathematical modeling using stochastic differential equations, we formalize how competing objectives in agentic systems create systematic interference patterns that degrade safety properties. Our empirical validation across 800 experiments reveals three critical failure modes: (1) systematic security vulnerabilities from AI-generated code, with efficiency-focused prompting introducing memory safety issues in 42.7% of cases, while security-focused prompting paradoxically creates cryptographic vulnerabilities in 21.1% of cases;  (2) iterative degradation, where security vulnerabilities increase by 37.6% after just five feedback iterations; and  (3) knowledge dilution, where domain expertise degrades by 47% as irrelevant context accumulates.  Our multi-agent governance and security architecture, validated through industry partnerships, achieves a 40% reduction in post-deployment safety incidents while maintaining operational capability. These findings establish that safe, secure, and aligned agentic systems require architectural integration of governance mechanisms that address the dynamic, multi-objective nature of autonomous AI systems.

SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems

Safe agents can fail together. Multi-agent LLM systems (MAS) move information, state, decisions, and authority across principal boundaries, creating failures that local checks may miss. Without an execution-level view, a multi-agent setting can easily be mistaken for evidence of a genuinely multi-agent security effect. We thus systematize MAS security through an execution-centered analysis of 197 works, covering six interaction interfaces, four adversary positions, seven system-level risks, and eight recurring attack paths. We introduce an A-I-R framework that organizes attacks by adversary position, interaction interface, and resulting system-level risk, unifying otherwise fragmented attack mechanisms across MAS. We organize defenses through a five-part contract covering path target, observation, intervention, trust boundary, and recovery, and identify path closure and recovery as key challenges. We audit 44 evaluation and benchmark works and identify open challenges in isolating interaction effects, designing comparable and diagnostic metrics, supporting reuse across MAS designs, and evaluating open-system operation. Together, these findings motivate an interaction-aware view of MAS security: trace attacks end to end, test whether defenses close those paths, and evaluate system-level effects with appropriate counterfactuals.

Open Security Benchmark: Towards Autonomous Enterprise Cyber Defense

Open Security Benchmark is presented, a framework that benchmarks agentic AI on security posture management work and surfaces a curated enterprise environment that evaluates posture investigation across two modalities: text-to-SQL over a relational snapshot and each vendor's native API over a served instance of the same environment.

Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response

Cyber-capable AI agents combine language models with tools, memory, and execution environments to perform multi-step offensive-security tasks. Existing work separately measures cyber capability and catalogs attacks against agent components, but provides less guidance on containing a capable agent within the environments used to evaluate it. This review synthesizes five vulnerability classes at that boundary: multi-step offensive chains, objectives that conflict with sandbox boundaries, supply-chain and credential exposure, persistent command-and-control, and the speed of automated action. We use two separate preliminary incident records: the reported July 2026 Hugging Face/OpenAI evaluation breach and Anthropic's subsequent three-incident evaluation review. A comparative evidence protocol distinguishes record-specific factual claims from the shared systems lesson: the evaluation environment is itself part of the security boundary. Across the taxonomy and records, we examine controls for containment, privilege separation, provenance, and responder access, including the dual-use problem that defensive artifacts may also enable misuse. The review identifies practical priorities for evaluating cyber capability together with the security of the environment in which that capability is exercised.

Security of Foundation-Model-Powered Embodied Agents: Attack Surfaces, Attacks, Defenses, and Evaluation

This work presents a trust-boundary-centric survey of foundation-model-powered embodied-agent security, and shows that attack research is concentrated on multimodal perception and action interfaces, while defenses are especially concentrated on action-level and runtime protection.