Skip to content
Preprint

MuseFlow-TSE: Music-Aware Target Speech Extraction via Cross-Source Band-Splitting and Data-to-Data Flow Matching

Oct 2026 · 0 citations · 29 references
Computer Science

TL;DR

MuseFlow-TSE, a discriminative deletion, generative insertion framework for music-bearing scenes, is presented, a music-aware cross-source band-split extractor that estimates target, interfering speakers, music, and noise under conservation and music-leakage constraints.

Abstract

Target speech extraction (TSE) isolates a speaker's voice given an enrollment utterance, but existing work is validated almost exclusively without music. Discriminative methods are content-faithful yet masking-bounded; generative methods improve naturalness but may hallucinate, drift in identity, and require many steps. We present MuseFlow-TSE, a discriminative deletion, generative insertion framework for music-bearing scenes. Stage-1 is a music-aware cross-source band-split extractor that estimates target, interfering speakers, music, and noise under conservation and music-leakage constraints. Stage-2 is a data-to-data flow refiner initialized by the discriminative estimate, negatively conditioned on interference stems, and run for 1--4 difficulty-adaptive Euler steps with an end-to-end fidelity objective.

View source

Similar papers

Preprint Oct 2026

Phonology-Aware Speaker Extraction Frontend for Audio LLMs

Target speaker extraction (TSE) is a key frontend for enabling audio large language models (LLMs) to recognize and understand a designated speaker in overlapping speech. However, multilingual mixtures exhibit heterogeneous extraction demands, causing fixed-depth systems to either under-process challenging inputs or was...

Yan-Xiang Chen, Hao-Yuan Deng, Jin-Yu Wang et al. · 0 citations
Review Sep 2026

Multimodal Target Speaker Extraction: Towards Unified Speaker Cues Across Modalities

A review of the current landscape and open problems in multimodal TSE from the perspective of auxiliary target cues drawn from multiple modalities and summarizes future directions concerning adaptive cue fusion, instruction-driven extraction, realistic evaluation, and trustworthy deployment.

Xin-Yuan Qian, Yang Zhou, Zi-Yang Jiang et al. · 0 citations
#natural language process... Preprint Nov 2026

Pandemonium: Longform Target-Speaker Speech Recognition with Cross-Speaker Interactions

Diarization-conditioned target-speaker ASR (TS-ASR), in particular Diarization-conditioned Whisper (DiCoW), is an accurate and data-efficient approach to multi-talker transcription. It has, however, three structural limitations: the model transcribes each speaker independently, operates on short (30 s) windows, and opt...

Alexander Polok, Dominik Klement, I. Medennikov et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Tap2Listen: Instant Target Switching for Streaming Audio-Visual Speaker Extraction

Streaming audio-visual target speaker extraction (AV-TSE) uses a target speaker's face to extract the speaker's speech from an input audio mixture in real time. However, existing systems fix the target speaker for an entire clip, while conversation demands frequent and immediate target switches. To address this problem...

Hans-Sol Park, Min-Jing Song, Zinu Kim et al. · 0 citations
Preprint Nov 2026

Speaker-Consistent Grounding for Confusion-Resilient LLM-Based Target Speech Extraction

Deterministic target-speaker extraction (TSE) tends to preserve observed linguistic content but may extract the interfering speaker. In contrast, autoregressive generative TSE improves perceptual quality, but is prone to content hallucination and speaker identity drift, which can be mitigated by model grounding with th...

Han-Yu Meng, Hao Shi, Zhan-Hong He et al. · 0 citations
#machine learning Preprint Oct 2026

SepRQ : Self-Supervised Speech Mixture Representation Learning via Mask-Free, Multi-Scale Source Separation

Self-supervised learning (SSL) is standard for speech representation learning, but mainstream models are designed around single-speaker audio, limiting their usefulness in multi-speakers scenarios. We present SepRQ, an open-source SSL framework that replaces masked prediction with a pseudo-source-separation objective o...

Séverin Baroudi, Hervé Bredin, R. Marxer · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.