A key strength of Proximal Policy Optimization (PPO) is its learned critic, which uses historical trajectories collected during reinforcement learning to estimate expected returns and reduce policy-gradient variance. However, we find that the critic is also a major source of instability in reinforcement learning for la...
Xuan-Yi Zhou, Qiu-Yang Mang, Huan-Zhi Mao et al.· 0 citations
It is shown that adding more agents is not a sufficient defense, because the adversary can simply scale with the group, and that allowing deceivers to coordinate privately can make them less effective.
Addison J. Wu, Jasin Cekinmez, Michel Liao et al.· 0 citations
The approach, LeAct (Learning to reason from Actions), optimizes this latent variable: the student samples candidate CoTs for each expert action, and the student retains those that measurably improve its own probability of recovering the action.