Skip to content

ReListen: Training-Free Evidence Localization in Hour-Long Audio with Frozen Audio Language Models

Nov 2026 · 0 citations · 38 references
Computer Science Engineering

TL;DR

ReListen is a training-free method in which a frozen audio language model finds its own evidence before answering, and its memory stays constant even for recordings too long to hear whole on one GPU, therefore, a frozen audio language model can search hour-long recordings by itself, without a transcript or additional training.

Abstract

Audio language models are meant to answer questions from the audio itself, but an hour-long recording rarely fits in their context, and the answer often depends on a few seconds of it. Transcribing the recording first makes it fit but discards sounds and voices, while systems that return to the audio must be trained for this behavior. We propose ReListen, a training-free method in which a frozen audio language model finds its own evidence before answering. In a Jev-style scan, the model scores each 20-second window by the probability that it helps answer the question, and it then answers from the highest-scoring minute. This minute overlaps the annotated evidence far more often than transcript retrieval, for 88.5% of VoiceGiraffe questions. Without any change, the same scan also grounds events on LAT-Bench as well as a model trained for temporal grounding. Answering from it matches or exceeds hearing the whole recording with a small fraction of the audio, and on VoiceGiraffe it exceeds a transcript of the whole recording by 9.2 points. The selected evidence is also worth hearing rather than reading. Given the same excerpts, reading their transcript matches listening on speech content, but listening is clearly more accurate on questions about sounds and voices. Moreover, ReListen answers faster than every transcript-based system, and its memory stays constant even for recordings too long to hear whole on one GPU. Therefore, a frozen audio language model can search hour-long recordings by itself, without a transcript or additional training.

View source

Similar papers

Preprint Oct 2026

Heard But Not Said: Prompting Understates Temporal Information of Musical Moments in Frozen Large Audio-Language Models

Large audio-language models (LALMs) understand music, yet struggle to locate a described moment in time. We ask whether this reflects missing temporal information or a bottleneck in how the time span is extracted. Using a paired-query probe built from Slakh2100 stem activations, we attach a state query (e.g., ``only pi...

Eunsik Shin · 0 citations

LEAP: Learned Block-wise Evidence Retrieval for Long Audio-Video Perception

Hour-scale audio-visual question answering is constrained by a context dilemma: dense whole-recording encoding rapidly exhausts context limits, whereas uniform temporal compression severely dilutes fine-grained acoustic and visual evidence. We introduce LEAP, a framework where the model retrieves its own evidence witho...

Ju-Yi Lin, Zhi-Qiang Lao, Jia-Li Cui et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Do Audio LLMs HEAR Pain? Tracing Acoustic Evidence from Speech to Language Reasoning

HEAR is introduced, a four-stage diagnostic framework that traces pain-related information from the speech signal to the model's prediction, and applies it to seven closed-source and open-weight LALMs, indicating that current LALMs encode paralinguistic information in early layers that is largely unavailable when the m...

Wen-Qi Li, Xiang-Jue Dong, Sulaiman Vesal · 0 citations

What Did the MLLM Hear? Token-Level Spectro-Temporal Grounding for Audio MLLM Explainability

STAG is introduced, to the authors' knowledge the first post-hoc framework for token-level spectro-temporal grounding of captions generated by audio-based MLLMs, and provides behavioral support for the faithfulness and selectivity of the explanations.

Lucia Cascone, Valeria Fraenza, Michele Nappi et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.