Jul 2026· Annual International ACM SIGIR Conference on Research and Development in Information Retrieval· pp. 5394-5397· 0 citations· 15 references
Computer Science
TL;DR
It is argued that the IR community is well-positioned to advance this emerging problem setting, which is referred to as AgentSearch, by developing principled methods for representing, indexing, retrieving, and ranking AI agents and tools under multi-dimensional relevance criteria.
Abstract
As AI agents and tools increasingly carry out everyday tasks on behalf of users, the digital ecosystem is rapidly filling with agents that offer overlapping or complementary functionalities, raising a fundamental challenge: how can users, developers, and orchestrating systems effectively identify, compare, and select the most appropriate agents or tools for a given task? While rooted in traditional information retrieval (IR), agent search introduces new challenges because the retrieved entities are executable systems rather than static information artifacts, and their suitability depends on capabilities, behaviors, constraints, uncertainty, and task-dependent performance. We argue that the IR community is well-positioned to advance this emerging problem setting, which we refer to as AgentSearch, by developing principled methods for representing, indexing, retrieving, and ranking AI agents and tools under multi-dimensional relevance criteria. The AgentSearch Workshop aims to bring together researchers and practitioners from academia and industry to explore the challenges and opportunities of agent and tool search, including agent discovery, capability representation, retrieval and ranking models, evaluation methodologies, and issues related to safety, fairness, personalization, and explainability. To support concrete progress, the workshop will also host an exploratory AgentSearch Challenge that provides a shared experimental setting for ranking agents given a task. The workshop will adopt an interactive format featuring breakout discussions, poster and spotlight sessions, invited talks, and collaborative activities, fostering active engagement and cross-disciplinary exchange beyond a traditional mini-conference structure.
The field of information retrieval has been rapidly transformed by AI technologies, especially large language model (LLM) agents with strong reasoning, planning, and conversational capabilities. These AI agents have improved how information is retrieved, processed, and personalized across search and recommendation systems. Despite these advances, important challenges remain, including relevance, bias mitigation, real-time response, and data security. This workshop aims to bring together researchers and practitioners to discuss recent advances, practical applications, and future directions of AI agents in information retrieval, while encouraging collaboration and knowledge exchange within the community.
Qingsong Wen, P. Mehrotra, Yongfeng Zhang et al.· Proceedings of the 32nd ACM...· 0 citations
EMBL AI Librarian is introduced, a knowledge layer that upgrades the Europe PMC interface for AI agents that improves performance across a range of tasks: literature synthesis, claim verification, open-domain question answering, and downstream biology tasks such as protocol questions and sequence manipulation.
Luigi Sigillo, M. Silvestri, Francesco Tabaro et al.· arXiv.org· 0 citations
The term agent in artificial intelligence lacks a standard definition, complicating the evaluation, comparison, and reproducibility of AI agent research. We address this ambiguity through a survey organized around five dimensions of agenticness: environmental interaction, learning and adaptation, autonomy, goal-directed behavior, and temporal coherence. For each dimension, we examine how the underlying capability has been conceptualized across prior work and synthesize the metrics, benchmarks, and evaluation frameworks used to assess it. This review provides a structured account of the current landscape of agent evaluation, highlighting both established approaches and areas where evaluation remains limited or inconsistent. We additionally introduce the Agent Compendium, a public-facing digital resource that organizes and extends the evaluation methods identified through this review. Together, the survey and compendium provide a common structure for evaluating and comparing agent capabilities across AI systems, supporting more reproducible research, clearer communication, and more systematic study of artificial agents.
In this paper, we analyze the reasoning trajectories of a variety of DRAs and show that existing agents often suffer from reasoning stagnation: the majority of iterations contribute little or no improvement to final performance, while agents lack awareness of their trajectories and are therefore ineffective at adapting their search strategies or determining when to terminate. To address this issue, we introduce a set of unsupervised signals and a Retrieval-Aware Agent Controller (RAAC), which assists the agent in selecting optimal actions at each stage of the research process. RAAC incorporates key information retrieval principles, namely search novelty and information coverage, resulting in more effective reasoning trajectories that improve overall performance while reducing unnecessary iterations, and consequently cost and latency. Specifically on BrowseComp-Plus and across a large set of DRAs, adding RAAC reduces the number of search calls by an average of 14, significantly improves the best-performing DRA on recall and accuracy, and achieves an accuracy gain of up to 10% (3% on average).
Heydar Soudani, Elisabeth Lingg, Faegheh Hasibi et al.· 0 citations
Despite their common goal of assisting users in information access, current Information Retrieval (IR) systems, such as search engines and recommender systems, are studied and deployed separately across application contexts, resulting in scattered user information and fragmented support for a user's task. Can we develop a single general intelligent system to unify those specific systems for assisting users with information access using multiple modes (e.g., search, recommendation, and conversation)? In this perspective paper, we present a vision for developing a general intelligent information agent (GIANT) that not only unifies the current specific systems for supporting information access but also goes beyond information access to provide personalized task completion for users. We propose to formalize GIANT generally as an agent interacting with its users to minimize their effort on finishing a task as well as the agent's resource overhead. We propose a probabilistic modeling framework for optimizing GIANT's interactions with its users and discuss how to estimate its four component models, including Situation Model, Content Model, Task Model, and User Model. We discuss how the framework can be refined to derive specific interaction strategies and implemented with a general Markov Decision Process (MDP) architecture. We further discuss how to evaluate GIANT using user simulation and conclude with an outline of some promising directions for future research.
Cheng-Xiang Zhai· Annual International ACM SIG...· 0 citations
This work introduces SearchOS, a system-level multi-agent framework that turns fragile, implicit search progress into explicit, persistent, and shared state, and introduces a Search Tool Middleware Harness that intercepts model and tool interactions to record grounded evidence and react to stalls or budget exhaustion.
