UTILMEM: Benchmarking Evidence Utilization in Long-Term Conversational Memory
UtilMem is introduced, a diagnostic benchmark comprising 1,717 instances across five domains, designed to evaluate four underexplored aspects of memory utilization: reasoning over dense histories, identifying implicitly relevant memories, synthesizing distributed evidence into summaries, analyses, or plans, and resisting interference from semantically similar distractors.
Peijun Qing, Fobo Shi, S. Vosoughi
· 0 citations