Skip to content

Safety in Self-Evolving Agents: A Survey

Sep 2026 · 1 citation
Computer Science

TL;DR

SAVER is introduced, a transition-centered framework in which Substrate locates reusable influence, Adaptation captures how it changes, Violation identifies compromised safety attributes, Exposure marks where failures become observable, and Response assesses containment, repair, or revocation.

Abstract

Large language models (LLMs) exhibit strong general capabilities, yet their parameters typically remain fixed after deployment, limiting learning from new interactions. In open-ended environments, this motivates self-evolving agents that continually update reusable state-including model parameters, memories, tool definitions, skills, and workflows-from data, feedback, and accumulated experience. This shift changes the safety problem: once experience becomes reusable state, past events become future causes, and information harmless in one context may later influence decisions with greater persistence, authority, or scope. Self-evolving agent safety therefore asks not only whether a response is aligned or an action authorized, but whether safety properties survive the accumulation, generalization, and cross-context reuse of locally useful experience. We introduce SAVER, a transition-centered framework in which Substrate locates reusable influence, Adaptation captures how it changes, Violation identifies compromised safety attributes, Exposure marks where failures become observable, and Response assesses containment, repair, or revocation. Our survey reveals that failures need not originate from harmful information: legitimate state can become unsafe when adaptation expands its persistence, authority, or scope beyond the conditions under which it was valid. Existing work provides comparatively strong evidence for admission, retrieval, activation, exposure, and local containment, but much less for descendant repair and evaluation after adaptation resumes. We therefore argue for longitudinal evaluation that traces unsafe influence to its originating transition, verifies repair across descendants, and tests whether it can re-emerge under continued evolution.

View source

Similar papers

The Right Lesson at the Right Step: Deriving Control Updates for Self-Evolving Agents

Results show that self-evolving agents should place experience inside the control flow, rather than only store it as text, and show how learned conventions must reach the right execution step on long tool-use environments where learned conventions must reach the right execution step.

Yun-He Su, Zi-Yi Dong, Tong Yu et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Self-Evolving Agent Harness

Harness self-evolution revises an agent's software scaffolding, including prompts, memory, workflows, and executable components, while keeping model weights fixed. We formalize a feedback-driven process in which accepted revisions persist across runs. The framework separates update targets from proposal and admission m...

Chu-Yang Wei, Hao-Xiang Guan, Ming-Hang Zhu et al. · 0 citations
#artificial intelligence Preprint Oct 2026

EvoBelief: Falsifiable World Models for Self-Improving LLM Agents

Self-evolving LLM agents typically improve how they act through accumulated experience, reflection, or skills. Yet even procedurally capable agents can fail when their beliefs about environment dynamics are incomplete, incorrect, or outdated. We study a complementary axis of self-evolution: evolving those beliefs durin...

Janet Wang, Jiang Guo, Yi Xiang et al. · 0 citations
#artificial intelligence Preprint Aug 2026

Safety Does Not Compose: Non-Decaying Loop State for Autonomous LLM Agents

LoopHarness is presented, which restores a persistent, non-decaying safety state at the loop level at the loop level, and gives a complete evaluation protocol on native Agent-SafetyBench tasks with paired clean and attacked episodes, an outer-state attack suite whose decisive evidence exists only across iterations, per...

Chenmin Wu, H. Jia, Yang Liu et al. · 0 citations

Experience Funnel: A State-Policy Alternating Loop for Self-Evolving Agents

Experiments across diverse agent benchmarks show that \textit{Experience Funnel} consistently improves agent capability over state-only evolution and policy-internalization approaches, while progressively converting useful explicit experience into autonomous policy competence.

Wen-Bo Gao, Zhao-Mou Song, Zhi-Yuan Ji et al. · 1 citation

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.