Automated Red Teaming using Reinforcement Learning

Jul 2026

A systematic evaluation of large language models for autonomous cyber defense

Overall, the results show that LLMs can produce competitive defense policies without fine-tuning but require manually engineered prompts, and their higher variance and slower response times could render them unsuitable for some real-world scenarios.

Thibaut Jacques · Applied intelligence (Boston... · 0 citations

2026

A Reinforcement Learning-Driven Adversarial Attack Methods With Dynamic Perturbation Optimization

Pretrained programming language models (PLMs) have shown strong performance in software engineering tasks, but remain vulnerable to adversarial attacks. Traditional methods rely on narrow, static perturbation strategies, leading to poor diversity and a combinatorial explosion of the search space when multiple operation...

Lulu Wang, Xiaoyi Wang, Xiaoning Du et al. · IEEE Transactions on Reliabi... · 0 citations

Jul 2026

GPT-Red: Automated Red Teaming via Self-Play at Scale

GPT-Red, an automated red-teaming agent that is trained to discover novel prompt injection attacks against frontier LLMs, is introduced, a scalable self-play algorithm where the model is tasked with attacking a diverse population of simultaneously-trained defender agents.

Eric Wallace, Christopher A. Choquette-Choo, Nikhil Kandpal et al. · arXiv.org · 2 citations

Conference Open access 2026

RLShield: Dynamic Jailbreak Detection for LLMs via Reinforced Adaptive Learning

RLShield is a dynamic jailbreak detection framework that employs reinforcement learning for adaptive threshold selection and incorporates three key innovations: a dynamic retrieval and LLM-based rewriting module to simulate diverse adversarial contexts; a cross-layer representation analysis to pinpoint safety-critical...

Zhao Tong, Pengfei Yang, Yimeng Gu et al. · Annual Meeting of the Associ... · 0 citations

#machine learning Preprint Aug 2026

Trident : How to Break Deep Reinforcement Learning Cyber Defenses (Agentic)

Empirical evaluations reveal a fundamental brittleness in existing defenses: with a single trainable 7B planner, Trident reduces blue agent defensive performance by an average of 522% compared to static red agent baselines while autonomously discovering emergent behaviors such as decoy avoidance and adaptive state prio...

Ryozo Masukawa, Ian Bryant, Armita Kazeminajafabadi et al. · 1 citation

Preprint Aug 2026

TRACE: Trajectory Aware Reasoning for Multi-Turn Adversarial Conversation Evaluation

This work introduces Trace, a multi-turn defense with trajectory-aware structured reasoning that balances usability and safety, and trains Llama-3.1-8B-Instruct with SFT and GRPO under a multi-component reward that jointly optimizes helpfulness on benign prompts and robustness against jailbreak attempts.

Md Messal Monem Miah, Adrita Anika, Zhiyuan Yu et al. · 0 citations