Audio captioning benchmarks typically evaluate a single unstructured free-text description of an audio scene, providing limited insight into how a model fails to understand polyphonic audio. A plausible caption may omit a source, or conflate multiple sources into one, or assign an attribute of one source to another, an...
Sripathi Sridhar, Yujin Kim, Giovana Morais et al.· 0 citations
This work introduces TAC2, a foundation model that casts timestamped audio captioning as composable functions to construct granular, speaker-attributed acoustic timelines directly from in-the-wild audio, and extends it with expressive speech attributes to obtain TAC2-S, a unified model that yields strong spoken audio u...
Swapnil Bhosale, Prem Seetharaman, Ke Chen et al.· 0 citations
Generative TSE model PromptSep-Speaker is proposed, a generative TSE model based on latent diffusion that unifies four interchangeable cues: speaker temporal order, transcript, enrollment audio, and speaker caption that achieves state-of-the-art performance over TSE and universal source separation baselines, improving...
Yu-Tong Wen, Ke Chen, P. Seetharaman et al.· 0 citations
CrossEdit is presented, a unified omni-modal editing model for images, audio, and video that performs AV movie scene edits zero-shot through cross-modal transfer and proposes AV-FES, a metric that jointly scores instruction following and consistency.
William Chen, Prem Seetharaman, Ke Chen et al.· 0 citations