Listen, Critique, and Refine: RL-Based Self-Refinement for Instruction-Following Speech Synthesis
This work training a LALM with reinforcement learning to reason over its own speech output generates a draft speech as a form of audio-token reasoning, critiques its own generation by reflecting on the acoustic realization in text, and produces a refined version conditioned on both the first-pass speech and the critiqu...