Skip to content
Open access

CAAP: Context-Adaptive Adversarial Policy Learning Under Partial Observability

Oct 2026 · Computers · Vol 15, pp. 675 · 0 citations

TL;DR

CAAP (Context-Adaptive Adversarial Policy) is a black-box adversarial policy framework that represents recent interaction history with a recurrent Context Encoder, combines multiple action-value experts through a differentiable Soft Policy Router, and encourages early exploration through Context Coverage Exploration using an annealed KNN-based coverage bonus in the learned context space.

Abstract

Adversarial policies provide a practical means of stress-testing reinforcement-learning agents by inducing failure through environment interactions without modifying the deployed victim policy. Learning effective adversarial policies under partial observability is challenging because the attacker must act from incomplete observations without access to victim-side parameters, gradients, or internal states. Existing approaches often rely on instantaneous observability patterns or predefined partitions, which can discard temporal interaction information and provide limited guidance for exploring rarely encountered but potentially vulnerable contexts. We propose CAAP (Context-Adaptive Adversarial Policy), which is a black-box adversarial policy framework that represents recent interaction history with a recurrent Context Encoder, combines multiple action-value experts through a differentiable Soft Policy Router, and encourages early exploration through Context Coverage Exploration using an annealed KNN-based coverage bonus in the learned context space. Context Coverage Exploration is used only during training, leaving the Context Encoder, Soft Policy Router, and action-value experts as the deployed policy. In the primary particle-pursuit benchmark, CAAP achieves the lowest in-domain catch rate among the five internal methods and the lowest macro-average catch rate across eight zero-update transfer conditions. These results support the use of history-dependent Soft Policy Routing with training-only Context Coverage Exploration for black-box adversarial policy learning under partial observability.

Read PDF

Similar papers

#machine learning Preprint Nov 2026

RLGPA: A Reinforcement Learning-Guided Granularity Planning Attack for Decision-Based Sparse Adversarial Examples

Decision-based sparse attacks must reduce perturbation support using only the victim's predicted label. The best restoration strategy varies as support contracts: coarse removal works early, while tail states need finer, history-aware tests. We propose RLGPA, an online reinforcement learning method that schedules suppo...

Chao Zhou, Sheng-Hua He, Jia-Xin Mo et al. · 0 citations
#artificial intelligence Preprint Oct 2026

AgentSafeEvo: Adversarial Co-Evolution for LLM Agent Safety

This paper introduces AgentSafeEvo, a co-evolutionary reinforcement learning framework for agent safety built on an attacker-defender game spanning diverse tasks and attack types, and introduces learnability-guided task weighting, which uses within-task defender-reward variability to adjust each task's contribution to...

Chen-Xu Wang, Xiaoyu Wen, Han Qi et al. · 0 citations
#reinforcement learning Open access Oct 2026

Risk-Adaptive Behavioral Attenuation via Episode-Latched Policy Gating and Bounded Relearning for Deployed Reinforcement Learning Agents

Reinforcement learning behaviors can become harmful after deployment even when the policy is unchanged, a risk existing methods address mainly before or during training rather than through reversible correction. This paper proposes a risk-adaptive attenuation framework for a frozen context-conditioned policy encoding v...

Mehdi Samieiyeganeh, Tahir Mehmood, Parisa Bahraminikoo et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Vulnerability Analysis for Safe Multi-Agent Reinforcement Learning

Safe Multi-Agent Reinforcement Learning (SMARL) aims to learn cooperative control policies under prescribed safety constraints. Al- though existing SMARL methods focus on enforcing safety during policy optimization, their vulnerability remains largely unexplored. This paper investigates inference-time observation attac...

Tao-Zheng Zhu, Shi-Xiong Jiang, Fan-Xin Kong · 0 citations
#machine learning Preprint Oct 2026

When to Intervene? State-Aware Sparse Manipulation in Federated Reinforcement Learning

Federated reinforcement learning (FRL) enables distributed agents to collaboratively train decision-making policies, but its decentralized training process also exposes global policy learning to Byzantine manipulation. Existing poisoning attacks primarily focus on how to construct malicious updates, while trajectory-le...

Shu-Tong Zheng, Si-An Chen · 0 citations
#reinforcement learning Open access Oct 2026

Balancing adversariality and naturalness in autonomous vehicle testing scenario generation via multi-objective reinforcement learning.

Reinforcement learning (RL)-based adversarial scenario generation through dynamic agent-environment interactions has emerged as a promising approach for accelerating autonomous vehicle (AV) testing. However, existing RL approaches struggle to balance adversariality and naturalness because naturalistic driving behaviors...

Li-Jing Cheng, Yu-Zhi Chen, Si-Xuan Xu et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.