Faithful LLM-Assisted Question Answering for Technical and Academic Search

Abstract

Technology scouting and competitive intelligence increasingly rely on rapid access to scientific and technical evidence. Large language models (LLMs) are reshaping question answering by enabling users to express exploratory and evolving information needs in natural language. However, their usefulness is limited by hallucinations, weak calibration, poor abstention when evidence is missing, and the cost of keeping knowledge up to date. In such settings, the key question is whether LLM-generated answers can be trusted for decision support, that is, whether they are faithful to available evidence, transparent about uncertainty, and easy to verify. Recent work shows that hallucinations remain pervasive, while retrieval-augmented generation (RAG) has become the main strategy for grounding answers in external evidence. Yet even strong RAG systems often cite unsupported claims or generate plausible but fabricated references, especially under poor retrieval conditions. In response, prior work has examined citation faithfulness, decomposed generations into checkable units, and proposed factuality verification models and benchmarks. Still, the literature underexplores how to filter and validate evidence when sources are sparse, conflicting, or rapidly changing, as is common in technical and academic search. This thesis addresses that gap by studying faithful LLM-assisted question answering for technical and academic search as an end-to-end problem spanning collection construction, model adaptation, grounded generation, evidence verification, and user-centered evaluation. It is organized around five questions: how LLMs can support exploratory search beyond keywords; how retrieval and generation pipelines can produce trustworthy responses; how systems should behave when evidence is sparse or unreliable; how usefulness and trustworthiness should be evaluated; and which architectures and optimization strategies enable reliable real-world deployment. The thesis argues that truthful technical search requires explicit mechanisms for claim-level grounding, calibrated abstention, contradiction handling, and support-aware evidence presentation. Methodologically, the work follows a pipeline perspective in which datasets, models, and evaluation procedures co-evolve. The empirical setup combines public QA and RAG benchmarks with technical and scientific collections tailored to technology scouting scenarios. A central methodological contribution will be the design of scenario-based evaluation subsets targeting null-retrieval, conflicting evidence, noisy or weak-signal evidence, and exploratory information needs. On top of these resources, the thesis will compare single-pass RAG and more structured agentic pipelines while varying retrieval strategies, prompting policies, claim-evidence alignment procedures, quotation and citation behaviors, and model adaptation techniques such as instruction tuning, parameter-efficient fine-tuning, and reinforcement-based optimization. Evaluation will be multi-dimensional. In addition to answer utility and retrieval effectiveness, the thesis will measure citation faithfulness, support coverage, hallucination and false citation rates, abstention quality, uncertainty communication, and robustness under perturbed retrieval conditions. Whenever feasible, these automatic analyses will be complemented with analyst-oriented user studies that examine whether different evidence presentation formats help users reduce verification cost in realistic scouting workflows. Overall, the expected contribution is a principled and deployable framework for trustworthy natural-language access to scientific and technical information, combining faithful generation, verifiable evidence selection, and evaluation protocols tailored to real technical and academic search.