EvidenceRank: an Explainable Evidence Retrieval Framework for Conversational Agents
Abstract
This paper presents EvidenceRank, referred to as Glyph throughout this work, a desktop question-answering conversational agent and extractive summarization system for public-domain texts. The system addresses a practical retrieval problem: how to answer specific natural language questions from long literary, historical and scientific works without relying on labeled training data or external APIs. Error analysis identified recurring limitations in document and evidence scoring, including word-sense ambiguity, interference from structural metadata. The proposed system combines document ranking, query expansion, and an Anchor/Signal passage scoring method that separates topical alignment from informational intent. An optional retrieval-augmented generation layer using a locally hosted language model enables comparison between extractive and generative outputs. Our results suggest improved document selection and passage precision for question-answering systems used in conversational agents within the evaluated examples. The retrieval pipeline uses explicit scoring rules and runs locally.