This work proposes Convex Hull Neighborhood Smooth Dual Generalization (CSDG), which expresses the Bellman backup as an in-sample value target plus a CHN-local correction, and characterize the implicit policies induced by the idealized operators and give a conditional non-degradation criterion.
Abstract
Offline reinforcement learning (offline RL) can benefit from nearby out-of-distribution (OOD) actions, but estimation errors at these actions may be amplified by bootstrapping. Existing regularization and local-generalization methods control either the admissible OOD region or the influence of generalized targets, often through separate mechanisms. We propose Convex Hull Neighborhood Smooth Dual Generalization (CSDG), which expresses the Bellman backup as an in-sample value target plus a CHN-local correction. This formulation makes the generalized contribution explicit and separates it from the in-sample reference path. The correction is obtained by smoothing in-sample-oriented and OOD-oriented candidates sampled at different perturbation radii. A mixture coefficient lambda scales its contribution to each backup, while the recursive discount remains gamma. Under boundedness and fixed perturbation kernels, we derive an exact one-step correction identity, a time-varying iterate bound, and a fixed-point bound that depends only on the branch discrepancy at the fixed point. We further characterize the implicit policies induced by the idealized operators and give a conditional non-degradation criterion. The practical algorithm approximates these quantities using asymmetric bounded noise and expectile regression, without exact support classification or an additional pessimistic OOD penalty. Experiments on Gym-MuJoCo and AntMaze show strong aggregate performance and stable value estimation. Code is available at: https://github.com/YOUNG-fnxm/CSDG
Qualitative studies in a toy environment show that BADA generates mixed samples that preserve desirable policy smoothness while accurately reconstructing multimodal value distributions, and extensive experiments on limited offline datasets further demonstrate that BADA attains state-of-the-art performance across divers...
Gong Gao, Weidong Zhao, Xianhui Liu· IEEE Transactions on Knowled...· 0 citations
A general recursive framework that includes the Chambolle--Pock algorithm and related primal--dual splitting methods is developed, which proves finite-sample guarantees for both estimators and establishes a matched-Gaussian universality result beyond Gaussian designs.
A two-phase accelerated method that achieves, with high probability, an accelerated optimization contribution and smooth-subclass-optimal statistical dependence on accuracy, up to logarithmic and generalized-smoothness factors.
D. Dvinskikh, A. Gasnikov, A. Lobanov et al.· 2 citations
A convex combination of one-step and two-step residuals with a fixed mixing weight is considered to provide a natural bias--variance trade-off between approximation error under a restricted value-function class and the increased variance arising from multi-step importance weighting.
Divergence-based regularization and Sharpness-Aware Minimization (SAM) are two prominent approaches for improving generalization in deep learning, both motivated by robustness to perturbations. However, their relationship has remained largely unexplored. Building on classical second-order expansions of $f$-divergences,...
The central limit theorem and consistency of the fully transcript-based sandwich covariance and intrinsic Wald region under local differential privacy are proved and the predicted decrease in estimation error and near-nominal coverage under moderate privacy is supported.