Skip to content

The Right Lesson at the Right Step: Deriving Control Updates for Self-Evolving Agents

Sep 2026 · arXiv.org · Vol abs/2609.34988 · 0 citations · 51 references
Computer Science

TL;DR

Results show that self-evolving agents should place experience inside the control flow, rather than only store it as text, and show how learned conventions must reach the right execution step on long tool-use environments where learned conventions must reach the right execution step.

Abstract

Self-evolving agents improve future behavior by reusing past experience, typically as global prompts, memories, or reflections. Yet these mechanisms rarely control where experience takes effect. In long tool-use workflows, the same lesson may correct one decision but distract another, making experience reuse a problem of localized control rather than memory alone. We introduce EvoCUE (Evolution through Control Updates from Evidence), a framework for learning reusable control-program updates from completed agent executions. EvoCUE represents the agent as an explicit state-machine controller, whose nodes perform model or tool calls and whose edges define where control passes next. This makes the workflow editable at precise locations, so each learned update can specify what to add, where it acts, and when it applies. From completed trajectories, EvoCUE uses residual goals and observed execution traces to propose localized instruction or skill edits. Each candidate is evaluated at the point where it would act by resuming the parent and edited controllers from the same checkpoint and comparing their final outcomes. Accepted edits are compiled with applicability rules, confirmed on held-out tasks, and inherited by later executions. We evaluate EvoCUE on long tool-use environments where learned conventions must reach the right execution step. From a minimal AppWorld controller without benchmark-specific onboarding instructions, EvoCUE learns the missing task-completion convention and substantially improves success on Test-Normal and Test-Challenge. On PAST-Bench office workflows, EvoCUE transfers organizational requirements from prior episodes to later tasks, improving task-execution quality. These results show that self-evolving agents should place experience inside the control flow, rather than only store it as text.

View source

Similar papers

#artificial intelligence Preprint Oct 2026

Self-Evolving Agent Harness

Harness self-evolution revises an agent's software scaffolding, including prompts, memory, workflows, and executable components, while keeping model weights fixed. We formalize a feedback-driven process in which accepted revisions persist across runs. The framework separates update targets from proposal and admission m...

Chu-Yang Wei, Hao-Xiang Guan, Ming-Hang Zhu et al. · 0 citations
#artificial intelligence Preprint Nov 2026

LiveAsset: Learning Intent Assets from Corrective Interactions for Agent Runtime Control

Large language model (LLM) agents acting across web applications and online services often need multi-turn corrections to align their work with users'underspecified goals and preferences. In practice, users may identify problems in the current outcome without specifying the desired result, so learning user intentions r...

Guan Liu, Weihang Wang, Peng Zhang et al. · 0 citations
Preprint Nov 2026

Decision Context: Persistent Behavioral Learning for AI Agents

LLM agents do not learn from corrections across sessions. A user fixes the same mistake at the start of every conversation. Existing agent memory systems primarily handle facts, such as preferences, entities, and conversational history, rather than procedural knowledge. We present Forge Decision Context (forge_dc), a s...

Nima Vahdat, Pashootan Vaezipoor · 0 citations
#artificial intelligence Review Nov 2026

Review the Episodes, Refine the Lessons: Simple Memory Evolution for Embodied Agents

Recent studies of embodied agents based on frontier multimodal large language models (MLLMs) develop harness engineering around the MLLM, such as harnesses with post-trained policies and recursive self-improvement loops. Yet whether these additions exceed the native capability of the MLLM remains unexplored. We introdu...

Yi-Fei Yao, Yu-De Zou, Xing Gao et al. · 0 citations
Review Oct 2026

PAUSE: Search-Free Steering of Coding Agents Before Their First Source-Code Edit

Coding agents can invest substantial effort in repository exploration and testing yet still produce unsuccessful patches. Existing online steering methods review trajectories periodically or invoke an advisor when a monitor detects behavioral problems; frequent reviews are costly, and the absence of anomalous behavior...

Viral Patel, Bo-Yin Tan, Jason Xue et al. · 0 citations
Preprint Nov 2026

When Should an Agent Refresh? Adaptive Index Reuse for Efficient Long-Horizon Reasoning

Language agents read code, tool output, and dialogue in contexts that grow at every step. Dynamic sparse attention lets a small indexer choose the earlier tokens each query attends to, but the indexer still scores the whole prefix at every layer. Cross-layer reuse methods such as IndexCache rerun the indexer at only so...

Guang-Yuan Wang, Yi-Fan Zhang, Shi-Long Liu · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.