Skip to content

Augmenting Large Audio-Language Models with Frame-Level Grounding for Fine-Grained Temporal Perception

Sep 2026 · arXiv.org · Vol abs/2609.15215 · 0 citations · 28 references
Computer Science

TL;DR

This work augments the LALM with a dedicated frame-level grounding model while leveraging its semantic modeling capability to represent the event query, and shows that the grounding model can provide temporal evidence to support downstream reasoning.

Abstract

Large Audio-Language Models (LALMs) have substantially advanced general audio understanding, yet they remain limited in fine-grained temporal perception, particularly in precise event localization. Existing approaches primarily post-train LALMs to predict event boundaries as timestamp tokens. However, this generative formulation lacks explicit correspondence between the timestamp predictions and fine-grained acoustic evidence, limiting the precision and reliability of temporal localization. To address this issue, we augment the LALM with a dedicated frame-level grounding model while leveraging its semantic modeling capability to represent the event query. Specifically, the frozen LALM encodes the event query with audio as context, and the grounding model combines these query representations with fine-grained audio features to localize the target event at the frame level. Extensive experiments across diverse temporal grounding benchmarks demonstrate strong and consistent improvements over existing methods. Further evaluation shows that the grounding model can provide temporal evidence to support downstream reasoning.

View source

Similar papers

Open access Sep 2026

Probing Large Audio-Language Models for Compositional Understanding of Sounding Actions

Large audio-language models (LALMs) excel at understanding and reasoning tasks over atomic sound events, yet their ability to infer higher-level human activities from such fine-grained events remains largely unexamined. Everyday human actions and activities, such as setting a table, cleaning the house, or preparing a b...

Michel Olvera, Paraskevas Stamatiadis, Chang-Hong Wang et al. · 0 citations

What Did the MLLM Hear? Token-Level Spectro-Temporal Grounding for Audio MLLM Explainability

STAG is introduced, to the authors' knowledge the first post-hoc framework for token-level spectro-temporal grounding of captions generated by audio-based MLLMs, and provides behavioral support for the faithfulness and selectivity of the explanations.

Lucia Cascone, Valeria Fraenza, Michele Nappi et al. · 0 citations
Preprint Aug 2026

CoSTALA: Compositional Spatio-Temporal Audio-Language Alignment via Multi-Grain Hierarchical Contrastive Learning

Conventional audio language models (ALMs) have made significant progress in achieving alignment between auditory and textual representations, including recent explorations in spatial audio. However, in daily spatial scenarios, they still cannot effectively process multi-event audio sequences. Current approaches primari...

Peiwei Ren, Jin-Bo Hu, Fang Kang et al. · 1 citation
Preprint Aug 2026

Note-Level Temporal Grounding of Musical Concepts in Large Audio-Language Models

Large audio-language models (LALMs) demonstrate growing music-understanding capabilities, but whether their responses are grounded in acoustic evidence remains unclear. Musical language often involves abstract concepts whose acoustic evidence is difficult to define and evaluate precisely. We introduce MusicGroundingBen...

Kun Fang, Ziyu Wang, Ichiro Fujinaga · 0 citations
#artificial intelligence Preprint Sep 2026

Kairos: A Dataset for Fine-Grained Video-Language Modeling over Space, Time, and Dynamics

Kairos is introduced, a video dataset for video-language modeling with time-resolved annotations that supports fine-grained evaluation, long-range modeling and reasoning, instruction data construction, representation learning, and video generation.

Ruibo Ming, Lei Sun, De-Heng Zhang et al. · 0 citations
#machine learning Preprint Oct 2026

AdaLoop: Adaptive-Depth Latent Reasoning for Audio Language Models

Large audio language models answer questions about speech, sound, and music, yet their accuracy drops sharply on tasks that need fine-grained acoustic analysis. Judging which of two speakers has the higher pitch demands iterative signal-level reasoning that a content question does not. Current models spend the same com...

Seung-Woo Lee, Bo-Wen Qi · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.