FroLineR: Front-Line Response with Retrieval-Augmented Prompt-Engineered Reply Generation for IT Help Desks

Feedback-Guided Prompt Injection Defense in Retrieval-Augmented Text-to-Cypher Generation

Text-to-Cypher generator systems translate natural language questions into Cypher queries, enabling intuitive interactions with graph databases such as Neo4j and Amazon Neptune. Despite recent advancements in LLM-based Cypher query generation, the vulnerabilities of the known methods—such as prompt injection attacks—are not discussed in detail. In this paper, we employ a robust Retrieval-Augmented Generation (RAG) architecture tailored specifically for text-to-Cypher tasks, leveraging dense vector retrieval to enhance query generation accuracy. We propose a dynamic and self-corrective procedure with feedback-loop-based AI architecture with Large Language Models (LLMs) for near real-time validation and correction of generated queries. We create a systematic procedure for generating datasets specifically designed to assess prompt injection robustness. Comprehensive evaluations are conducted using a diverse set of LLMs, including GPT-4o, DeepSeek R1, Claude 3.5 Sonnet and Qwen 2.5 Coder 32B Instruct. Our evaluation results indicate substantial improvements in resiliency against prompt injection attacks compared to various benchmarks. It is demonstrated that the proposed solution outperforms various training-free prompt injection defense methods.

Design and Implementation of a Retrieval-Augmented Generation (RAG) Based Chatbot for Document Question Answering

: With the rapid growth of enterprise databases, personal cloud storage, and corporate digital archives, the number of unstructured text documents requiring manual analysis and reading has increased massively. Extracting relevant information from massive multi-page documents manually is extremely time-consuming and prone to human oversight. Traditional keyword-based search systems and earlier rule-based chatbots do not really work that well anymore because they lack deep semantic understanding and fail to synthesize context-aware answers to user queries. This paper mainly talks about how Retrieval-Augmented Generation (RAG) frameworks are making document-based question-answering smarter and more reliable than traditional models. It explores different types of RAG techniques like dense retrieval, vector database optimization, generative language model integration, and hybrid search frameworks... and explains how they are implemented in real-world intelligent systems. The paper also discusses what these methods are good at, where they struggle a bit, and how practical they actually are when it comes to handling complex, multi-modal documents. One important thing the study shows is that RAG-based document chatbots can achieve contextual response accuracy levels of more than 95% and can reduce language model hallucinations quite a lot. Still, some problems are not fully solved yet — especially things related to processing nested tables within documents, sub-optimal retrieval latency, and parsing highly technical schemas in real time. Because of this, future research is expected to focus more on Graph-based RAG (GraphRAG), agentic multi-step reasoning, and privacy-preserving federated retrieval systems that can scale up dynamically as large document corpora evolve.

When the Database Fails: Prompting LLM Dialogue Agents for Safe Recovery in Task-Oriented Dialogue

Large language models used in task-oriented dialogue often produce fluent but unsafe responses when backend database calls fail, return empty results, or surface mismatched information, inventing venues, confirmations, or booking details not grounded in the database. We study a lightweight prompting-based recovery approach that improves robustness without retraining or additional model calls. We compare three response strategies, including a guided recovery prompt conditioned on structured database status, across six open-weight model families (DeepSeek-R1, Gemma-2, Llama-3, Mistral, Phi-3, and Qwen-2.5) and four database conditions: empty result, wrong-domain retrieval, API error, and clean retrieval. Using fault-injected benchmarks built on two structurally different datasets, MultiWOZ 2.2 (5 domains) and SGD (20 domains), we find that naive agents hallucinate on 30.5% of failure turns on MultiWOZ and 20.9% on SGD. Our Guided-Retry strategy reduces hallucination by 50% on MultiWOZ (30.5 to 15.3%) and by 42% on SGD (20.9 to 12.2%) without retraining. However, residual hallucination remains substantial (6-37% across models), with wrong-domain failures the hardest case. Results are consistent across both datasets and all six model families, and human annotation shows substantial agreement while supporting the validity of the automatic commitment-safety metric.

An Intelligent PDF Question-Answering System; A Retrieval-Augmented Generation Approach

Due to the surge in digital document creation, intelligent systems that can effectively retrieve accurate information from unstructured PDF content have become a necessity. Current keyword-based search techniques lack semantic meaning, while large language models (LLMs), when utilized individually, produce hallucinated results when they are not contextually aware of document content. In this paper, we introduce a novel real-time question answering system for PDF content using a lightweight approach to Retrieval-Augmented Generation (RAG). Our system combines a HuggingFace sentence transformer model for semantic embeddings with a FAISS vector similarity search for efficient retrieval, leveraging a Groq-hosted LLaMA 3.3 70B model for context-grounded answer generation. The system is built using LangChain for orchestration and Streamlit for web-based deployment, and requires no GPU, no fine-tuning, and no pre-indexed corpus. Evaluation across five complementary metrics—Exact Match, Token-level F1, BLEU, ROUGE, and semantic similarity—demonstrates strong alignment between generated and reference answers, with a peak semantic similarity of 0.8709 and a mean Token F1 of 0.4701, confirming the viability of this approach for real-world document intelligence.

Contrastive Reflection for Iterative Prompt Optimization

LLM agents are becoming central to information retrieval: they issue retrieval queries, synthesize answers, and increasingly serve as judges for IR evaluation. Improving the prompts that control these agents is an optimization problem, but in applied IR settings it often looks less like blind search and more like debugging. Engineers need to know which behavior failed, which nearby behavior still worked, what distinguishes the two, and whether a prompt edit improves held-out quality without introducing regressions. We present Contrastive Reflection, an iterative prompt-optimization framework for agentic IR workflows. The framework starts from a task-centric quality definition: QA agents expose retrieval or reasoning traces, and grading agents expose dimension-level scores and rationales. These structured traces are used to identify error-anchored behavioral slices, add nearby successful examples from the same region, and ask a Teacher LLM to propose a targeted prompt edit. Candidate edits are accepted only when validation performance improves, optionally subject to regression checks. We instantiate the framework with a tree-based slice selector, but the contribution is the contrastive reflection loop rather than the tree itself. On a public HotpotQA retrieval-augmented QA setup, one tree-selected contrastive repair improves held-out exact-match accuracy from 51.4% to 60.4%. Failure-only and random-evidence variants improve less and break more previously correct examples. A light instruction-only comparison places the method near modern prompt optimizers: MIPROv2 reaches 59.4% and GEPA 57.0%. The result is an interpretable optimization loop for IR agents, aimed at making prompt repair more inspectable and validation-driven.

Prompt Template-Driven Large Model SQL Generation