Security and privacy challenges of RAG systems

Multimedia, RAG, and LLMs in High-Security Environments - a reference implementation for the AI2MMIR architecture

Privacy-Preserving RAG by Concealing Sensitive Information from External LLMs

This paper introduces the Sensitive Entity Alias Generator (SEAG), a privacy-preserving framework that empowers users to utilize powerful third-party generators without disclosing sensitive information and demonstrates the success of the SEAG framework.

RH-RAG: Trustworthy Long-Form Generation for Privacy-Constrained Settings

Evaluations across literary, financial, and legal domains demonstrate that RH-RAG consistently improves factual grounding, semantic coherence, and document-level alignment compared to standard and hierarchical RAG baselines, while achieving reliability competitive with proprietary cloud-based systems without compromising data privacy.

Mitigating Database Leakage in RAG Systems with Keyword-Grounded Fact Substitution

KFS-RAG is proposed, a defense that mitigates information leakage by reformulating the retrieved context by identifying a small set of influential keywords from the retrieved context via an attention rollout plus a causal perturbation mechanism.

Is External Database Protection Static in Retrieval-Augmented Generation? Rethinking Privacy Preservation under Dynamic Queries

A Prompt-Aware Dynamic Hierarchical Differential Privacy framework (PA-HDP) is proposed, which performs a prompt-aware risk hierarchy to dynamically assess privacy risks under different queries and applies adaptive sensitive entity replacement and exponential mechanism-based text selection to provide differentiated privacy protection while preserving semantic utility.

Privacy Preserving Information Retrieval: Defining Privacy Research Pillars for a Future Research Agenda

Advancements in computer science are raising concerns and preoccupations about the privacy of users' data submitted to and used by Information Retrieval (IR) systems. IR systems, such as search engines, integrate new generative information access pipelines that implement effective and efficient document retrieval and answer generation. However, critical challenges arise in Privacy-Preserving IR (PPIR): Are such data leaking personal information or being used to train generative systems? Are current privacy solutions sufficient to guarantee user privacy and limit such information leakage? Are users' queries and retrieved documents protected throughout the entire retrieval process? How has the privacy threats landscape changed, and in which directions should the IR and Privacy research community investigate to address such new risks? In this perspective paper, we provide initial answers to these questions, analysing state-of-the-art solutions for protecting user privacy when accessing information and highlighting areas of concern. We propose a new PPIR research agenda to address the unsolved problem of private data use and access. The agenda includes novel privacy research pillars aimed at addressing objectives grounded in gaps in the literature, user survey findings, and structured interviews with experts from research, industry, and regulatory bodies. By defining these privacy research pillars, we advise the IR community to pursue research toward a more resilient privacy direction that can address future challenges stemming from rapidly advancing technology eager for user data.