Skip to content

CART: Closed-Loop Adaptive Red Teaming for Large Language Models

Sep 2026 · arXiv.org · Vol abs/2609.27336 · 0 citations · 44 references
Computer Science

TL;DR

CART turns red teaming from a one-time checklist into a continuous, adaptive, and auditable search for model and agent weaknesses, suggesting that contextual adaptation can reveal weaknesses that direct prompt replay does not exercise.

Abstract

Automated red teaming often replays a fixed set of prompts, which measures known risks but cannot learn from failures found during testing. We present CART (Closed-Loop Adaptive Red Teaming), a framework that uses each result to guide what it tests next. CART begins with broad risk coverage, follows weaknesses that emerge, keeps new probes diverse, and records the evidence and source of every finding. It separates the Challenger that creates tests, the Target being tested, which may be a text-only model or a bounded tool-using agent, and the Judge that evaluates the results, allowing these roles to be studied independently. Across three evaluation families (Frontier, JAH, and Agentic), CART discovers more failures and higher average risk than static seed replay for every Target with an available baseline. The gains extend to tool-mediated agent tests, suggesting that contextual adaptation can reveal weaknesses that direct prompt replay does not exercise. These results describe what the test policies discover, not how often failures occur in real deployments. We also find that Challenger-Judge choices affect the evidence uncovered, highlighting the need for role separation and independent review. Overall, CART turns red teaming from a one-time checklist into a continuous, adaptive, and auditable search for model and agent weaknesses.

View source

Similar papers

#artificial intelligence Preprint Oct 2026

Same Feedback, Different Answer: Measuring Run-to-Run Instability in Frontier-Model Customer Feedback Analysis

A repeat-run evaluation framework that aligns semantically equivalent categories and focuses on two operating metrics: theme churn, the normalized change in the returned category set, and volume disagreement, the change in counts for categories that persist shows that taxonomy grounding produces more consistent and rep...

Viraj Bagal, Raviraja Ganta, Prabhath Chellingi · 0 citations
#artificial intelligence Preprint Oct 2026

PROBE: Learning to Audit Policy Compliance in Tool-Using LLM Agents

This work proposes PROBE (Policy Rule Observation via Behavioral Exploration), an algorithm for training LLM-based auditors that interact with deployed agents as realistic users and probe them for policy violations through multi-turn, tool-mediated dialogue, and formalizes agent auditing as a two-player Markov game.

Kshitij Mishra, Abhijith Sharma, Nils Lukas et al. · 1 citation
#artificial intelligence Preprint Sep 2026

KC-Bench: A Dynamic Interactive Benchmark for Evaluating Knowledge Conflicts in LLM Agents

KC-Bench is introduced, a controlled multi-turn benchmark for measuring model-level behavior across world-knowledge conflicts, input inconsistencies, and multi-source temporal conflicts, and provides a reproducible diagnostic for developing conflict-aware reasoning and execution safeguards.

Yaxing Lyu, Sheng-Jie Zhou, B. Toh et al. · 2 citations
Review Oct 2026

Rethinking Verification in Agent Harnesses

Agent harnesses enable large language models to act through tools, observe the results, and iteratively work toward a task goal. Verification checks and revises agents'work to improve output quality. Yet there is limited guidance on how harnesses should organize this effort. We study verification as an axis of test-tim...

Zheng-Xin Zhang, Er-Kang Zhu, Yin Lin et al. · 0 citations
#artificial intelligence Review Oct 2026

Structural Baselines Matter for Agent Failure Detection and Attribution

An LLM agent using tools can fail silently. It stops before the task is done, raises no error, and answers fluently. Two kinds of monitor address this, one deciding whether a run failed and the other naming the step where failure became unavoidable. Much of the reported performance on both tasks comes from regularities...

R. A. Naagarajan, Stefan Streif · 1 citation

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.