Experience Retrieval Compression for Continual Adaptation of LLM Agents
TL;DR
Structured Experience Compression (SEC), a lightweight method that distills each trajectory into a four-field unit encoding the task type, key decisions, failure reason, and repair action, is proposed, suggesting potential suitability for long-lived deployment settings.
Abstract
Retrieval-augmented experience replay is an emerging approach for the parameter-free adaptation of large language model (LLM) agents at inference time. However, injecting raw multi-step trajectories into the agent's context is costly and risks flooding the prompt with irrelevant details as the experience pool grows. We propose Structured Experience Compression (SEC), a lightweight method that distills each trajectory into a four-field unit encoding the task type $(T)$, key decisions $(D)$, failure reason $(F)$, and repair action $(R)$. At inference time the top- $k$ units closest to the current task are retrieved and appended to the prompt. Experiments on ALFWorld and WebShop show that SEC outperforms rawtrajectory retrieval by 7.7 percentage points on ALFWorld and by an absolute reward gain of 0.056 on WebShop, while consuming only 37% of the context tokens. SEC also degrades more gracefully when the pool contains noisy or mismatched records, suggesting potential suitability for long-lived deployment settings.