The Entropy-Scaled Trust Region (ESTR), which scales each token’s off-policy deviation by its local entropy, requiring no auxiliary forward passes or explicit version-switch detection, consistently outperforms existing asynchronous methods and achieves the best train-inference consistency.
BRACE is proposed, an anchored Bellman-residual correction for stale value models that bounds the correction horizon to a prefix of policy tokens and anchors a constant-weight Monte-Carlo tail beyond it, which separates policy correction from reward propagation.
Guan-Qun Zhao, Zi-Jun Xie, Bin-Bin Zheng et al.· 0 citations
This work proposes Autonomy-of-Heads (AoH), a data-free method that identifies retrieval and streaming heads from the spectral geometry of query-key projections that retains 96.5% of Full Attention performance on average while reducing prefill and decode latency by up to 41.4 and 66.0%, respectively, and KV-cache memor...
Ye-Han Yang, Jun-Yuan Shang, Yang Li et al.· 0 citations
Group-Reflective Self-Distillation (GRSD), which derives capability-aligned and outcome-discriminative guidance from the policy's own verified rollouts, and refines turn-level credit assignment by modulating outcome-based advantages while preserving the verifier-determined learning direction.
