Skip to content
Open access

Schema-Conditioned Memory Evidence Ranking for Underspecified Tool Calls: A Controlled Study on Mem2ActBench

Oct 2026 · Electronics · Vol 15, pp. 4688 · 0 citations

TL;DR

SCAR–Pair improves selected-tool memory evidence ranking within the evaluated setting, and separates schema-expanded candidate generation from parameter-level scoring and integrates request, tool, field, and cross-encoder evidence through query-balanced pairwise learning.

Abstract

Large language model (LLM) agents often need historical conversations to recover argument values omitted from current tool requests. This study introduces SCAR–Pair, a two-stage ranker for memory retrieval after tool selection. It separates schema-expanded candidate generation from parameter-level scoring and integrates request, tool, field, and cross-encoder evidence through query-balanced pairwise learning. On a reconstructed Mem2ActBench bank containing 19,661 conversations and 391 evidence-bearing tasks, SCAR–Pair achieved 0.450 Recall@5 and 0.365 nDCG@5 under stratified five-fold evaluation, compared with 0.396 and 0.325 for a matched request-only pairwise ranker. Component-aware intervals and adjusted tests support these task-weighted gains in the ToolACE-dominated mixture. Controlled comparisons support the complete schema-view scoring configuration over request-only scoring on the same candidate pool but do not isolate correct-schema semantics or individual field components as independent contributors. The pairwise advantage over pointwise training is also not significant. Across the tested readers, higher retrieval quality does not yield a significant tool-call accuracy advantage over the strict request-only control. SCAR–Pair therefore improves selected-tool memory evidence ranking within the evaluated setting.

Read PDF

Similar papers

#artificial intelligence Preprint Oct 2026

Correspondences as Decisions: JevNexus for Decision-Centric Schema Matching

Schema matching increasingly uses generative language models to rerank retrieved column candidates, although the underlying task is a bounded correspondence decision. We present JevNexus, which combines typed pairwise decisions with schema/instance evidence and invokes listwise refinement only when the evidence disagre...

Run-Ze Li, Han-Chen Wang, Ying Zhang et al. · 1 citation
Book Open access Nov 2026

Support Selection Survives under Official-Context RAG: A Same-Backbone Boundary Study

We study bounded Agentic retrieval-augmented generation (RAG) for knowledge-intensive question answering. The task is to answer multi-hop or domain questions from auditable external evidence under a fixed inference budget. Existing Agentic RAG evaluations often vary the model, retriever, corpus interface, controller, t...

Chuan-Heng Ma, Yi-Zhao Wang, Sheng-Yang Li · 0 citations
Open access Nov 2026

Budgeted Two-Stage Pre-Validation for Retrieval-Grounded Machine Learning Pipelines: A Group-Held-Out Evaluation

Retrieval-grounded machine-learning pipeline synthesis must reconcile code templates with package versions and resource budgets. We study a two-stage pre-validator adapted from concrete dependency-satisfiability and resource-prediction mechanisms: an inexpensive retrieval-profile margin ranks candidate manifests; a bud...

Maya Jenkins, Thomas Baker, Theo Hernandez · 0 citations
Open access 2026

EviGraphRAG: Constraint-Verified Retrieval-Augmented Question Answering over Event Knowledge Graphs

EviGraphRAG is proposed, a ranker-agnostic reliability layer that separates candidate ranking from exact verification and explicit abstention, and exact conjunction over represented event constraints and linked source-record connectivity can improve selective reliability, while parser quality and candidate retrieval re...

Yu-Teng Sun, Yang Su, Xu-An Wang · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.