Extensive evaluations show that R2T2 brings streaming recognition close to the base model's full-context quality while delivering stable text with low emission lag, and its gains extend beyond the post-training languages, demonstrating a practical path from full-context ASR to stable streaming.
Abstract
Streaming speech recognition is increasingly consumed by machines, such as translators and voice agents, that act on each token as it appears. A token that is later revised costs wasted work or a wrong action. In this work, we present Confucius4-R2T2 (short for Real Real-Time Transcription), a stable-prefix streaming ASR model that never modifies its earlier output while maintaining high accuracy and low latency. R2T2 is obtained by post-training Qwen3-ASR-1.7B on 1,500 hours of Chinese and English audio while freezing its pre-trained audio encoder and projector. The adaptation is based on the concept of Longest Stable Prefix (LSP), a new notion that we introduce in this work. We prove that LSP is monotone under the prefix order, consistent with the full-audio output, and maximal among labels that equal a whole earlier output. Learning from LSP labels converts a full-context model into a stable-prefix streaming one without any architectural change. Extensive evaluations show that R2T2 brings streaming recognition close to the base model's full-context quality while delivering stable text with low emission lag. Its gains extend beyond the post-training languages, demonstrating a practical path from full-context ASR to stable streaming. We open-source the model weights and code at https://huggingface.co/netease-youdao/Confucius4-R2T2 and https://github.com/netease-youdao/Confucius4-R2T2.
In this paper, we describe Cake By VPBank’s system submitted to Task I - Multilingual Conversational Speech Recognition of the 2nd MLC-SLM Challenge, reaching 15.84% time-constrained permutation word error rate (tcpWER) on the blind 21-variant, 14-language evaluation set. Rather than change the model architecture, we a...
T. Huu, Nhat M. Nguyen, Huan Ho Kha Viet· 2nd Workshop on Multilingual...· 0 citations
Real-time conversational agents require speaker diarization that streams and runs on a CPU. Most systems apply an utterance-level speaker encoder to short, heavily overlapping chunks, which wastes computation and leaves the model optimized for the wrong task. We instead turn a state-of-the-art speaker recognition archi...
Speech large language models (speech LLMs) perform well on automatic speech recognition (ASR) when sufficient paired speech-text data is available, but their performance degrades in low-resource settings. A cascaded pipeline that performs speech-to-phoneme (S2P) conversion followed by phoneme-to-grapheme (P2G) conversi...
AWED, a word-level emission-delay metric defined relative to the acoustic end of each word, is introduced and post-train a DSM recognizer with GRPO under a reward that scores transcription accuracy and measured delay jointly, advancing the accuracy--latency Pareto frontier of streaming ASR without architectural change.
Diarization-conditioned target-speaker ASR (TS-ASR), in particular Diarization-conditioned Whisper (DiCoW), is an accurate and data-efficient approach to multi-talker transcription. It has, however, three structural limitations: the model transcribes each speaker independently, operates on short (30 s) windows, and opt...
Alexander Polok, Dominik Klement, I. Medennikov et al.· 0 citations
This work introduces TAC2, a foundation model that casts timestamped audio captioning as composable functions to construct granular, speaker-attributed acoustic timelines directly from in-the-wild audio, and extends it with expressive speech attributes to obtain TAC2-S, a unified model that yields strong spoken audio u...
Swapnil Bhosale, Prem Seetharaman, Ke Chen et al.· 0 citations