Skip to content
Preprint

FRAME: Factored Retrieval via Attribute Readouts for Object-Centric Scene Memory

Sep 2026 · 0 citations · 29 references
Computer Science

TL;DR

FRAME is proposed, which turns language into query-relevant attribute weights, uses learned readouts to estimate per-attribute evidence from object embeddings, and ranks objects by aggregating this evidence according to the query, showing that persistent object attributes can be exposed as composable evidence for accurate and efficient multi-attribute retrieval.

Abstract

Language-guided robots need persistent scene memories to follow instructions, revisit objects, and resolve references to objects encountered over time. While much of language-guided scene-memory retrieval has emphasized spatial or relational references, many everyday object references specify objects by multiple persistent attributes, such as category, material, size, or surface appearance. We formalize this problem as attribute-compositional retrieval, where a fixed object-centric scene memory is queried with natural language to retrieve the object satisfying the requested attributes. To investigate this capability directly, we introduce a controlled evaluation protocol with fixed scene memories and attribute-defined targets, separating retrieval from perception and annotation ambiguities. We then propose FRAME, which turns language into query-relevant attribute weights, uses learned readouts to estimate per-attribute evidence from object embeddings, and ranks objects by aggregating this evidence according to the query. Across held-out scenes and object assets, FRAME outperforms representative scene-memory retrieval baselines while reducing post-decomposition object scoring to lightweight matrix-vector computation. These results position attribute-compositional retrieval as a complementary scene-memory capability for language-guided robots, showing that persistent object attributes can be exposed as composable evidence for accurate and efficient multi-attribute retrieval.

View source

Similar papers

#artificial intelligence Preprint Oct 2026

Have I Scene This Before? Spatially Grounded Conversational Memory for Complex Queries in Egocentric Assistants

Egocentric assistants must connect what users say with what they see across long interaction histories. We formalize this challenge as Spatially grounded Conversational Reasoning (SpaCR): cross-scene, recall-oriented, and counterfactual spatial queries that combine user-stated facts with geometric evidence. Direct visi...

Jiazhou Liang, Liam Gallagher, Kiko Chen et al. · 0 citations
Preprint Oct 2026

Learning to Retrieve: Internalizing Memory Retrieval for Video World Models

Video world models aim to generate explorable, 3D-consistent scene videos conditioned on camera trajectories. Existing approaches often rely on external memory systems that explicitly retrieve previously observed content to mitigate scene drift during long-horizon generation. However, these auxiliary memory pathways op...

Jia-Kui Hu, Tai-Lai Chen, Yu-Qi Pan et al. · 0 citations
Preprint Sep 2026

Structured Spatio-Temporal Evidence Graphs for Open-Vocabulary Object Retrieval in Videos

Open-vocabulary object retrieval in videos requires answering free-form object queries under bounded query-time cost. Existing index-based systems typically store independent frame-level regions and retrieve them with vision-language similarity, which is effective for appearance queries but mismatched with predicates w...

Jing-Dan Wang, Chuanyou Li · 0 citations
Preprint Sep 2026

A Scene Language Model for Open-Vocabulary Scene Mapping

Results show that a persistent open-vocabulary 3D scene map can be maintained directly by a single vision-language model using only a lightweight text representation.

Adam Lilja, Fabio Hübel, Si-Ming He et al. · 0 citations
Sep 2026

Finder: Agentic Closed-Loop Object Finding for Embodied Grounding

Finder is presented, an agentic closed-loop object-finding primitive for embodied grounding that maintains a typed loop state that links query-conditioned planning, scoped evidence gathering, candidate verification, and accept/continue/abort control.

Shi-Xiong Xu, Zhi-Yuan Chen, Song Ding et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.