2025· Neural Information Processing Systems· pp. 130641-130673· 0 citations· 78 references
Computer Science
TL;DR
This work states that existing visual generative models are not yet ready for RL due to the following two fundamental drawbacks that undermine the foundations of RL.
Abstract
Reinforcement learning (RL) has become an indispensable post-training step for unlocking the full potential of Large Language Models (LLMs). Its core motivation is to incentivize the model’s inference trajectory via a reward model, effectively balancing the exploration–exploitation trade-off in scenarios where collecting exhaustive input–output ground-truth pairs is infeasible. This motivation naturally extends to visual generation, where perfect alignment between an image and a textual prompt is inherently ambiguous and often unattainable. However, existing visual generative models are not yet ready for RL due to the following two fundamental drawbacks that undermine the foundations of RL:
This work proposes VEG (verbal ϵ -greedy), a novel framework that leverages external feedback as a dynamic control variable to explicitly balance exploration and exploitation within the semantic space and achieves superior accuracy compared to standard RL baselines.
Yongchang Hao, Jie Hao, Yongsheng Mei et al.· 0 citations
Noise-Contrastive GRPO is introduced, which injects scale-calibrated Gaussian noise into the last hidden layer of the prompt-encoding pass for half of each rollout group, branching those rollouts from a displaced departure state and integrates into a standard RLVR pipeline as a ~50-line change to the inference engine.
Michael M. Jerge, Joseph Pelczar, J. Downes· 1 citation
This paper explores post-training reinforcement learning (RL), specifically GRPO, to directly align autoregressive perception models with their evaluation metrics, and designs an RL framework that addresses perception-specific challenges: reward design for set-structured outputs and multi-head sampling control.
Sofian Chaybouti, Yasser Dahou, N. Huynh et al.· 0 citations
This work proposes CritiqueDriveVLM, a novel unified three-stage framework internalizing reasoning directly into the VLM, and proposes Latent Thought Distillation to overcome the latency bottleneck.
Zhaohong Liu, Hao Ye, Xianlin Zhang et al.· 1 citation
This work presents a novel world model formulation where the reward prediction only depends on a subset of structured, symbolic components of the whole latent state, and demonstrates the strong generalisation properties of this approach over purely neural methods.
Isidoro Tamassia, Lennert De Smet, Giuseppe Marra· 0 citations
VisPlay is introduced, a self-evolving RL framework that enables VLMs to autonomously improve their reasoning capabilities from massive unlabeled image data and establishes a scalable path toward self-evolving multimodal intelligence.
Yicheng He, Chengsong Huang, Zongxia Li et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
