RRPO: Reference-Relative Policy Optimization with Stratified Conditional Rollouts
Across verifiable reasoning, open-ended generation, and post-SFT settings, RRPO remains competitive with verifier-based optimization, improves over weakly supervised baselines, and provides additional gains after supervised fine-tuning.