A unified post-training framework that equips pretrained text-to-speech models with natural-language control over segment-level emotion and duration is proposed, highlighting post-training as a practical approach to extending existing speech synthesis models.
Abstract
Audiobook narration, conversational agents, and audiovisual dubbing require fine-grained control over emotion and pacing within an utterance, yet many text-to-speech (TTS) systems remain limited to utterance-level conditioning. We propose a unified post-training framework for natural-language segment-level emotion and duration control. Supervised fine-tuning (SFT) establishes instruction following, while group relative policy optimization (GRPO) further improves control accuracy using emotion, duration, content, and speaker rewards. Without modifying the pretrained architecture or adding inference-time control modules, our method improves fine-grained controllability while preserving intelligibility and speaker identity, demonstrating post-training as an effective way to extend pretrained TTS models.
A controllable multi-speaker dialogue TTS framework that formulates synthesis as critique-driven iterative refinement, and unifies instruction-following synthesis and natural-language-guided attribute editing, enabling correction of expressive errors without full regeneration.
Kang-Xiang Xia, Xin-Fa Zhu, Hang-Rui Hu et al.· 0 citations
VoiceWeaver addresses the problem of adding expressive and environmental controls to a speech generator with structured label prefixes and staged emotion-tone-event training in a shared text-audio model, while embedding decorrelation and attribute dropout regularize conditioning.
Xiao-Su Su, Yun Cao, Yi-Ping Ni et al.· 0 citations
Deep learning-based zero-shot speech synthesis has achieved substantial progress in speaker generalization, but stable modeling remains challenging in fine-grained emotional scenarios. Existing systems often process textual and emotional conditions through shared or closely coupled pathways, which may introduce interfe...
Yan Zhu, Yu Wang, Yi-Jin Zhou et al.· PLoS ONE· 0 citations
In the most data-scarce conditions, ISL-trained models outperform single-pass pseudo-labeling and further approach fully supervised performance, demonstrating that gradient-based ISL is an effective solution to expressive label scarcity in low-resource TTS.
Nicholas Sanders, G. Henter, Simon King et al.· 0 citations
This work training a LALM with reinforcement learning to reason over its own speech output generates a draft speech as a form of audio-token reasoning, critiques its own generation by reflecting on the acoustic realization in text, and produces a refined version conditioned on both the first-pass speech and the critiqu...
Chee-En Yu, Yi-Cheng Lin, Sung-Feng Huang et al.· 1 citation
Dynamic speaking style adaptation in multi-turn multimodal interaction remains a major challenge for text-to-speech (TTS) systems. Existing context-aware TTS (CTTS) methods typically map dialogue context to speech in an end-to-end manner. Such implicit modeling makes contextual style decisions difficult to supervise, w...
Wen-Jie Tian, Kang-Xiang Xia, Jing-Bin Hu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
