This work analyzes a single-loop, entropy-regularized Natural Actor-Critic algorithm under compatible linear function approximation, and introduces an Exponential Translation mechanism that maps the regularized gap to the unregularized one up to an exponentially decaying tail.
DigenRL is introduced, a disaggregated RL framework for diffusion-based generative LLMs that supports flexible resource allocation, accommodates heterogeneous GPUs, and facilitates efficient task scheduling and an elastic trainer-assisted generation approach to enable the trainer GPU resources to dynamically assist in...
Si-Jie Wang, Zhengyu Qing, Zhi-Qiang Tan et al.· arXiv.org· 0 citations
LeanGRPO is presented by restructuring the data-parallel layout and introducing two recompute-free training schedules for trajectory-logprob diffusion RL, which achieves up to 1.83x end-to-end speedup while preserving the original optimization objective.
Si-Jie Wang, Zhi-Qiang Tan, Xin-Rui Yang et al.· 0 citations
BiDiRL, a hybrid time-space multiplexing architecture for asynchronous, disaggregated RL designed to reduce resource idleness, is presented, including a hot-switch runtime that enables rapid switching between rollout and training resources with negligible overhead and a static, scheduling-aware planner based on time-pe...
Zhiqiang Tan, Maoxin Wang, Sijie Wang et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.