Two counter-intuitive findings -- PII masking improved generation quality, and a structural"anchor distillation"scheme made SQL hallucination impossible by construction are reported, along with a reproducible recipe for full embedder fine-tuning in 8 GB of VRAM.
Abstract
Public institutions hold large volumes of sensitive documents and support tickets that cannot leave the premises, ruling out cloud-hosted language models entirely. We report on RAGAL, a retrieval-augmented assistant for the technical-support team of AFIR, the Romanian Agency for Financing Rural Investments, built and operated under three hard constraints: zero data egress (no external API calls, even for synthetic data), a read-only mandate (the assistant drafts, humans execute), and a single 8 GB consumer laptop as the only development and training machine. Over a Romanian-language corpus of ~25,000 chunks -- 15,073 resolved support tickets and internal normative documents -- we show that the highest-leverage investments were retrieval engineering and retriever fine-tuning rather than a larger generator: hybrid dense-sparse retrieval with intent routing raised our internal evaluation from 62% to 81%, and fine-tuning the bge-m3 embedder on real ticket data improved recall@10 from 0.663 to 0.850 (MRR 0.489 to 0.684) after 72 minutes of training. We document a general pitfall: single-domain fine-tuning silently degraded retrieval on the untouched document domain below the stock baseline, detected only after building a per-domain evaluation set and repaired with locally generated queries (GenQ). We report two counter-intuitive findings -- PII masking improved generation quality, and a structural"anchor distillation"scheme made SQL hallucination impossible by construction -- along with a reproducible recipe for full embedder fine-tuning in 8 GB of VRAM. Finally, since zero egress also rules out a cloud judge, we describe a substitute: a 744B-parameter model run on CPU, too slow to serve interactively but affordable in overnight batch, used as a second opinion whose limits we quantify. We release the sanitized pipeline scripts for institutions facing similar data-locality constraints.
This work proposes READ (Reliable Embedding-free Agentic Document-search), in which an agent reads the raw document through three deterministic operations -- normalized lexical search, structural navigation, and bounded span reads -- exposed over the Model Context Protocol, so a trajectory is a replayable audit trail, not an opaque similarity score.
This paper presents a technical synthesis of RAG for trustworthy enterprise assistants and describes the end-to-end pipeline in full: document chunking, embedding, vector indexing, retrieval, cross-encoder re-ranking, and grounded generation with inline citations.
Bini P. B.· International Journal of Inf...· 0 citations
Banana (Bounded Adaptive Navigation Architecture for Nested Archives) replaces opaque vectors with plain-text Markdown indices built via one-time VLM page transcription, eliminating both a separate Optical Character Recognition (OCR) engine and a vector store.
Anup K. Roy, R. Upadhyay, Animesh Rameshbhai Panara et al.· Annual International ACM SIG...· 0 citations
It is proved that, under an honest-majority assumption and a representation-level separation condition, RAGSentinel exactly recovers a poison-free majority-sized context.
Yueyang Quan, Anjun Gao, Yu Xia et al.· 0 citations
The idea of context is no longer considered secondary in the construction of language-model systems. With the use of local Retrieval-Augmented Generation, even a tiny modification of the prompt or the context might produce another set of retrievals, citations, and ultimately different answers; however, in practice, tests are often performed with only one version of the question. In this work, we suggest a local context-engineering framework for exploring perturbation robustness, reproducibility, and budgetfriendly assessment in one unified pipeline. Local RAG is built on several small teaching packs for the models; perturbations are introduced to the queries, each experiment is recorded in capsule format, and a gate mechanism based on the decision tree is used to judge if the whole perturbation suite can be skipped, minimized, or run. As a result, the augmented capsule-derived data set contains 3,570 perturbation rows, out of which 2,619 labeled rows are used for retraining. In this labeled subset, BLEU scores are available in 446 cases, answer perplexity and semanticjudgment scores are available in 1,668 cases, citation overlap is available for all rows, and retrieval overlap is available for 2,099 rows. The final decision tree obtains 0.915 ROC-AUC, 0.891 PR-AUC, 0.913 accuracy, 0.863 F1 score, and expected savings of 76.5% on the held-out augmented test split. A more conservative threshold setting lowers the stable-risk value from 0.084 to 0.062, while reducing expected savings to 57.4%. Thus, we provide a local study pipeline to examine the effect of perturbations on answer stability, contradictions, robustness, and budget-friendly evaluation.
Rahul Reddy Gangapuram, William B. Andreopoulos· International Conference on...· 0 citations
This paper presents a Hebrew-first local LLM chat agent that combines Retrieval-Augmented
Generation (RAG), citation-aware document answering, controlled web search, and full right-toleft (RTL) user interaction. Unlike cloud-only assistants, the default response path operates
locally, supporting privacy, predictable operating costs, and deployment in environments where
documents should remain on-premises. The system integrates document upload and indexing,
hybrid semantic and lexical retrieval, reranking, source citation construction, streaming
responses, and PDF export. The revised manuscript positions the system against local RAG
baselines and Hebrew-capable cloud alternatives. It clarifies that the current evaluation is a
preliminary system-oriented assessment focused on grounded Hebrew document question
answering, lookup, summarization, citation behavior, and latency. The main contribution is a
practical architecture for private, transparent, Hebrew-oriented RAG assistance rather than a new
foundation model.
Michael Sirkovich, M. Domb· International journal of adv...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
