Skip to content

What Is This Memory Useful For? Measuring Task-Conditioned Utility in Long-Term Agent Memory

Nov 2026 · 0 citations · 18 references
Computer Science

TL;DR

A public, versioned Conditional Memory Utility Dataset of paired measurements and costs is provided to evaluate memory compression by the task utility it preserves alongside its costs.

Abstract

A stored experience may answer one future question and offer little help with another. We measure this task-conditioned utility rather than assigning each memory a single importance score. A task-blind writer stores a memory without seeing the later task; a frozen reader then answers that task with and without the memory, using identical filler. A cost ledger records the resources used to form, store and expose the memory. On single-call factual-recall tasks built from LongMemEval-V2 web-agent memories, the same memory helps more when it contains information needed for the answer. Its mean paired reward gain on these answer-bearing task pairs exceeds that on matched-control pairs by $+0.33$ (95% interval $+0.21$ to $+0.46$). The tested task-blind summaries and structured records yield smaller gains than Raw, including at fixed total context; a budget sweep is consistent with retention of answer-bearing content contributing to the difference. The bearing--control contrast remains positive in a new session with randomized memory position and under a second reader, although individual memory--task labels change. Generic retrieval performs above chance but misses most screen-defined answer-bearing memories. The factual-recall gains do not extend to the tested software-engineering and scientific-coding benchmarks, where even raw memories have no measurable effect. We provide a public, versioned Conditional Memory Utility Dataset of paired measurements and costs. The practical takeaway is to evaluate memory compression by the task utility it preserves alongside its costs. Task-conditioned measurements make this comparison possible; their stability must also be checked.

View source

Similar papers

#natural language process... Preprint Oct 2026

The Geometry of Agent Memory: Forecasting Retrieval-Bound Memory Failures, and Characterizing Its Limits

We study whether geometric relationships between conversation turns, memory texts, and queries can predict answer correctness before generation. We evaluate four memory implementations with dense or hybrid retrieval and one observation-summary comparator. We embed these texts in a shared space. A logistic classifier us...

I. Zhechev, Radoslav Ralev, Chris Coleman et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Recall What Is Missing: Composing Evidence from Long-Term Agent Memory

Under a limited context budget, a long-term agent must recall not merely memories that are individually relevant, but evidence that is collectively sufficient for the current query. Conventional memory access retrieves, scores, and truncates memories, often spending multiple context slots on redundant facts while leavi...

Qi Feng, Ji-Cong Fan · 0 citations
#artificial intelligence Preprint Oct 2026

Can Agents Forget When Asked? A Benchmark for Memory Unlearning in LLM Agents

Long-term memory has become a core component of large language model (LLM) agents. As agents accumulate personal information over long-term use, users increasingly ask them to delete sensitive or outdated facts. A memory system should honor such requests. It should remove exactly what was requested and keep the rest of...

Bo-Fan Xie, Chao-Kun Wang, Guan-Chen Ge et al. · 0 citations
Preprint Oct 2026

From Memory Hits to Answerable Evidence: Query-Conditioned Memory Compilation for Long-Term Agent Memory

Long-term agent memory requires recovering answer-relevant evidence dispersed across turns and sessions. Existing systems increasingly use structured memories and adaptive retrieval, yet often pass retrieved or expanded units directly to the reader, coupling memory addressing with evidence presentation and leaving the...

Shang-Ze Li, Yan-Yu Yao, De-Rong Xu et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Causal Memory Policy: Making Memory Utility Identifiable by Intervening on Retrieval

Memory-augmented large language models must decide which memories to retain, and recent systems do so by estimating each memory's effect on task performance. However, these estimates rely entirely on retrieved memories. When a memory is never retrieved, store-level interventions produce identical outcomes, leaving its...

Arman Behnam, Bing-Hui Wang · 1 citation · ⚡1

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.