Interactive models increasingly hand complex reasoning and tool use to a slow brain in the background, leaving a realtime front model free to sustain full-duplex dialogue. Yet the user may refine, cancel or replace a request while its task runs, so a plan or result that made sense a moment ago goes stale. Each new utte...
Yu-Xiang Wang, Sheng-Bo Cai, Zhi-Xian Zhao et al.· 0 citations
OmniCapBench (Omni-Video Caption Benchmark), a benchmark that reframes audio--visual caption evaluation as a deep-structured diagnostic framework, is introduced, providing a fine-grained roadmap for omnimodal development.
Zhongyu Yang, Jia-Le Tao, Rui-Tao Chen et al.· 1 citation
Model intelligence and fast response jointly shape the quality of interaction with speech language models, yet remain difficult to achieve together. Explicit chain-of-thought (CoT) improves reasoning and audio understanding, but generating intermediate reasoning tokens delays responses. Describing fine-grained acoustic...
Yu-Xiang Wang, Kun-Yu Feng, Yuan-Cheng Wang et al.· 2 citations
Gander is presented, a native multimodal duplex interaction model that builds on MiniCPM-o 4.5 and is further adapted for realtime interaction with an asynchronous agent loop and is released together with its models, code, and data to facilitate further research and development in the community.
Orantqing, Shengpeng Ji, Jun-Long Tong et al.· 3 citations
Full-duplex dialogue systems, which listen while speaking, must distinguish a completed turn from a pause within a turn and an interruption that requests a turn from a brief acknowledgment or speech addressed to a third party. Yet existing conversational corpora provide limited control over these events and limited lab...
Matthew Z. Sun, Vinay Kothapally, Meng Yu et al.· 0 citations
AuK, an open-source foundational model that unifies speech generation and editing through a common interface of natural-language instructions and audio context, is introduced and leading performance on zero-shot and instruction-controlled speech generation and general instruction-guided editing is demonstrated.
Zi-Yang Ma, Zhi-Kang Niu, Wen-Ming Tu et al.· 3 citations· ⚡1
Spatial-Omni is a lightweight method that implements SO-Encoder to inject First-Order Ambisonics (FOA) spatial audio into existing Omni LLMs as an independent modality, without modifying their original audio encoders.
In this work, we present Gander, an end-to-end model that unifies omni perception, realtime interaction, and agentic capabilities within a single framework. In contrast to turn-based conventional paradigms, Gander continuously receives streaming inputs across multiple modalities, including video, speech, and text, enab...
Orantqing, Sheng-Peng Ji, Jun-Long Tong et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
