This work studies how policy improvement is composed through multi-step proximal policy improvement (MPI), which re-centers each proximal objective on the preceding policy, and identifies improvement composition as a design choice alongside regularization strength, with distinct effects from horizon subdivision and add...
Role-Adaptive Policy Optimization (RAPO), which adapts policy-update coefficients according to their roles in value learning and execution, is proposed, which shows improvements over both base algorithms.
Offline reinforcement learning (RL) must reconcile two competing requirements: policy updates should stay near dataset-supported actions to keep value estimates reliable, yet meaningful gains often require moving beyond the behavior distribution. We develop a geometric view of offline actor updates by modeling policies...
The proposed PathBridger is a hierarchical offline GCRL method that explicitly connects subgoal selection to short-horizon execution, and constructs a state-space bridge toward the selected intermediate endpoint and decodes it into a short executable action chunk using an inverse dynamics model.
