TreeHCA: Hindsight-Guided Exploration and Credit Assignment for Multi-Turn LLM Agents
Outcome-supervised group-relative policy optimization (GRPO) assigns the same advantage to every token in an LLM agent trajectory, leaving useful and unhelpful decisions undifferentiated. Hindsight credit assignment (HCA) can refine this feedback, but its estimates rely on trajectories containing informative alternativ...
Debajoy Mukherjee, Jeremy Carleton, Sushil Vemuri et al.
· 0 citations