Ming-Yu Zhao

UniStream: Multi-Expert Residual Vector Quantization for 48 kHz Causal Streaming Audio Coding

We present UniStream, a fully causal 48 kHz neural audio codec for streaming speech, music, and environmental sounds. At its core is Multi-Expert Residual Vector Quantization (ME-RVQ), which replaces the single shared codebook in each residual quantization layer with four expert codebooks controlled by a deterministic...