SafetyMem: Adaptive Jailbreak Defense via Dual-Component Safety Memory
TL;DR
This work proposes Safet-yMem, a novel framework that secures LLMs through a dual-component safety memory sys-tem, and introduces an adversarial memory expansion mechanism that proactively generates challenging variants to solidify these memories.
Abstract
Current defenses for Large Language Models (LLMs) often suffer from a “memory gap”: parameter-modifying methods are computationally rigid, while inference-time filters cannot retain or reuse defense knowledge across interactions. To address this, we propose Safet-yMem , a novel framework that secures LLMs through a dual-component safety memory sys-tem. SafetyMem consists of Semantic Safety Memory (SSM) , which consolidates diverse jailbreak attempts into a structured knowledge base of attack patterns, and Episodic Safety Memory (ESM) , which maintains an evolving set of procedural rules refined from historical detection failures. Unlike static defenses, Safe-tyMem allows the model to “remember” and adapt to emerging adversarial strategies without parameter retraining. To further enhance robustness, we introduce an adversarial memory expansion mechanism that proactively generates challenging variants to solidify these memories. Experiments on standard and stealthy jailbreak benchmarks show that SafetyMem substantially reduces attack success rates while preserving efficiency and interpretability, consistently outperforming state-of-the-art base-lines across multiple LLMs.