Skip to content
Preprint

Evo-RL: Iterative Robot Reinforcement Learning from Selective Experience

Oct 2026 · 0 citations · 22 references
Computer Science

TL;DR

Evo-RL is an iterative reinforcement learning method that combines human interventions with advantage-guided experience selection to iteratively refine the policy on a real-world high-precision manipulation task.

Abstract

Existing real-world reinforcement learning methods improve robot policies by combining human corrections with autonomous rollouts. However, accumulated experience varies in its value for policy improvement, and even successful trajectories can contain inefficient or redundant behavior, motivating more selective experience reuse. To this end, we propose Evo-RL, an iterative reinforcement learning method that combines human interventions with advantage-guided experience selection. Evo-RL uses human corrections and experience with positive estimated advantages from successful trajectories to iteratively refine the policy. To support learning from these different experience sources, we introduce RW-RL-Dataset, which spans over 1,000 hours across four robot embodiments. On a real-world high-precision manipulation task, Evo-RL improves attempt success rate over RECAP by 30.46\%. Our code, model checkpoints, and dataset are publicly available.

View source

Similar papers

Sep 2026

Real-World Reinforcement Learning with MPC Scaffolding for Dexterous Manipulation

A framework that uses sampling-based model predictive control (MPC) as scaffolding for real-world dexterous RL, providing structured prior experience and task-directed guidance during learning without human demonstrations or corrective actions is proposed.

Emek Barış Küçüktabak, Karankumar Patel, Zhao-Dong Yang et al. · 3 citations
Sep 2026

ReF-HIL: Shaping the Critic around Human Action Neighborhoods for Efficient Human-in-the-Loop Reinforcement Learning

Human-in-the-loop reinforcement learning (HIL-RL) offers a promising route to efficient training of robotic manipulation policies by combining autonomous learning with human demonstrations and online corrections. However, insufficient use of successful human experience in value learning prolongs costly real-world train...

Shao-Yin Luo, Song Wang, Shi-Bo Xia et al. · 0 citations
#artificial intelligence Preprint Oct 2026

ACCRUE: Continual Off-Policy Reinforcement Learning for Robot Foundation Models

Reinforcement learning (RL) post-training improves robot foundation models (RFMs) beyond their demonstrations, but real-world deployment demands more: learning new tasks efficiently while retaining previously acquired behaviors. Existing continual RL methods on top of VLAs mitigate forgetting but rely on on-policy upda...

Changyeon Kim, Kimin Lee, Yashraj Narang et al. · 0 citations

Demonstration-Free Success-Probability Reward Learning for Generalist Robot Policies

This work introduces eVTA, which learns success probabilities from mixed-quality policy rollouts through temporal-difference-style bootstrapping, without expert demonstrations or intermediate annotations, and introduces RL with Evolving Rewards (RLER), a closed-loop framework that adapts eVTA using newly collected roll...

Duo Wu, Hai-Feng Wang, Rongwei Lu et al. · 2 citations

Learning from Mixed-Quality Deployment Experience for Robot Manipulation

Predictive Action Chunk Learning first learns a predictive chunk-level critic that evaluates temporally extended action sequences and augments temporal difference learning with future latent prediction, providing richer supervision for long-horizon value estimation.

Yan-Gang Ren, Yu-Jie Yan, Zi-Rui Li et al. · 0 citations
Sep 2026

RoboFFT: Finetuning generative robot policy via online reinforcement learning with forward process

Generative models, such as diffusion and flow-based models, have shown strong promise for robot policy learning by capturing complex and multimodal action distributions from demonstrations. However, policies trained solely with imitation learning often suffer from imperfect demonstrations and distributional shifts, whi...

Yu Li, Sheng-He Hu, Yu-Han Wang et al. · 1 citation · ⚡1

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.