Robust reinforcement learning (RRL) aims to develop a robust policy that maintains stable performance across diverse environments characterized by an uncertainty set. This set consists of perturbed environments derived from a nominal (training) environment that generates samples, thereby capturing potential discrepancies between training and real-world conditions. Recently, an adjacent uncertainty set has been introduced, providing more realistic perturbations compared to conventional formulations. Despite its solid theoretical foundation, the existing sample-based implementation of the robust Bellman update suffers from limited scalability and practical applicability in real-world scenarios. In this brief, we present, for the first time, scalable RRL algorithms that overcome these challenges by leveraging expectile regression. Extensive experiments demonstrate that the proposed methods significantly enhance the robustness of state-of-the-art (SOTA) RL algorithms while maintaining a practical computational cost comparable to strong off-policy baselines. In particular, the proposed methods exhibit up to a 23.6% average improvement in robustness under environmental perturbations over SOTA RL baselines while maintaining comparable computational complexity.
Ukjo Hwang, Songnam Hong· IEEE Transactions on Neural...· 0 citations
Offline reinforcement learning (RL) must reconcile two competing requirements: policy updates should stay near dataset-supported actions to keep value estimates reliable, yet meaningful gains often require moving beyond the behavior distribution. We develop a geometric view of offline actor updates by modeling policies as a probability manifold endowed with a chosen metric geometry. Under this lens, a broad class of offline actor objectives can be interpreted as a single proximal policy improvement step (SPI), i.e., an implicit discretization of a manifold gradient flow induced by a critic-defined energy. Building on this insight, we propose multi-step proximal policy improvement (MPI), a plug-in refinement mechanism that composes sequential re-centered proximal steps. MPI enables controlled policy improvement beyond dataset support while retaining proximal control at each refinement. The framework accommodates multiple policy geometries and admits practical instantiations for deterministic and diagonal-Gaussian policies. Experiments on D4RL benchmarks show that small numbers of MPI refinements improve strong offline baselines, including TD3+BC, ReBRAC, and IQL, on many tasks. Focused diagnostics further distinguish re-centered refinement from fixed-objective update scheduling and characterize limitations under critic error.
The proposed PathBridger is a hierarchical offline GCRL method that explicitly connects subgoal selection to short-horizon execution, and constructs a state-space bridge toward the selected intermediate endpoint and decodes it into a short executable action chunk using an inverse dynamics model.
Simulations show that HeLyMARL is the only method that sustains the throughput-fairness balance together with uninterrupted service throughout the horizon, outperforming conventional MARL, Lyapunov-based, and constrained MARL benchmarks without premature budget exhaustion.
Yeonseo Jeong, Wonhyeok Ko, Sungweon Hong et al.· 0 citations
This work proposes a lightweight and information-theoretically secure aggregation framework that securely computes the majority vote (MV) polynomial through single-round secure multiplication, ensuring end-to-end information-theoretic security under the honest-majority assumption while revealing only the final aggregated sign to the server.
