Skip to content
Preprint

Convex-Hull-Neighborhood Smooth Dual Generalization: Controlling Local Correction Propagation in Offline RL

Aug 2026 · 0 citations · 56 references
Computer Science

TL;DR

This work proposes Convex Hull Neighborhood Smooth Dual Generalization (CSDG), which expresses the Bellman backup as an in-sample value target plus a CHN-local correction, and characterize the implicit policies induced by the idealized operators and give a conditional non-degradation criterion.

Abstract

Offline reinforcement learning (offline RL) can benefit from nearby out-of-distribution (OOD) actions, but estimation errors at these actions may be amplified by bootstrapping. Existing regularization and local-generalization methods control either the admissible OOD region or the influence of generalized targets, often through separate mechanisms. We propose Convex Hull Neighborhood Smooth Dual Generalization (CSDG), which expresses the Bellman backup as an in-sample value target plus a CHN-local correction. This formulation makes the generalized contribution explicit and separates it from the in-sample reference path. The correction is obtained by smoothing in-sample-oriented and OOD-oriented candidates sampled at different perturbation radii. A mixture coefficient lambda scales its contribution to each backup, while the recursive discount remains gamma. Under boundedness and fixed perturbation kernels, we derive an exact one-step correction identity, a time-varying iterate bound, and a fixed-point bound that depends only on the branch discrepancy at the fixed point. We further characterize the implicit policies induced by the idealized operators and give a conditional non-degradation criterion. The practical algorithm approximates these quantities using asymmetric bounded noise and expectile regression, without exact support classification or an additional pessimistic OOD penalty. Experiments on Gym-MuJoCo and AntMaze show strong aggregate performance and stable value estimation. Code is available at: https://github.com/YOUNG-fnxm/CSDG

View source

Similar papers

#machine learning Open access Aug 2026

Improving Generalization and Robustness in Offline Reinforcement Learning via Boundary-Aware Data Augmentation

Qualitative studies in a toy environment show that BADA generates mixed samples that preserve desirable policy smoothness while accurately reconstructing multimodal value distributions, and extensive experiments on limited offline datasets further demonstrate that BADA attains state-of-the-art performance across divers...

Gong Gao, Weidong Zhao, Xianhui Liu · 0 citations
Preprint Aug 2026

Generalization Error Estimation for Primal--Dual Algorithms in Non-Smooth Regression

A general recursive framework that includes the Chambolle--Pock algorithm and related primal--dual splitting methods is developed, which proves finite-sample guarantees for both estimators and establishes a matched-Gaussian universality result beyond Gaussian designs.

Kai Tan, Pierre C. Bellec · 0 citations
Preprint Sep 2026

Localize, Restart, Accelerate: Stochastic Optimization under Generalized Smoothness

A two-phase accelerated method that achieves, with high probability, an accelerated optimization contribution and smooth-subclass-optimal statistical dependence on accuracy, up to logarithmic and generalized-smoothness factors.

D. Dvinskikh, A. Gasnikov, A. Lobanov et al. · 2 citations
#machine learning Preprint Sep 2026

Offline Policy Evaluation via Mixed Bellman Residuals and Adaptive Critic Representations

A convex combination of one-step and two-step residuals with a fixed mixing weight is considered to provide a natural bias--variance trade-off between approximation error under a restricted value-function class and the increased variance arising from multi-step importance weighting.

Amitakshar Biswas, Yu-Han Li, Ruo-Qing Zhu · 0 citations
#machine learning Preprint Sep 2026

Explaining f-Divergence-Based Regularization via Local Curvature and Sharpness-Aware Minimization

Divergence-based regularization and Sharpness-Aware Minimization (SAM) are two prominent approaches for improving generalization in deep learning, both motivated by robustness to perturbations. However, their relationship has remained largely unexplored. Building on classical second-order expansions of $f$-divergences,...

Nour Jamoussi, Marios Kountouris · 1 citation
#machine learning Preprint Sep 2026

Locally Private Inference for Riemannian Stochastic Optimization

The central limit theorem and consistency of the fully transcript-based sandwich covariance and intrinsic Wald region under local differential privacy are proved and the predicted decrease in estimation error and near-nominal coverage under moderate privacy is supported.

Xiao-Tian Chang, Yang-Di Jiang, Qi-Rui Hu · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.