Skip to content
Open access

Enhancing Auditory Reasoning in Large Audio–Language Models Via Supervised Fine-Tuning and Reinforcement Learning With Verifiable Rewards

2026 · IEEE Access · Vol 14, pp. 135822-135836 · 0 citations · 54 references

TL;DR

It is suggested that auditory reasoning can benefit from training strategies tailored to audio signals and objective answer verification, and they provide empirical evidence for applying RLVR to large audio–language models.

Abstract

The objective of this paper is to improve and analyze auditory reasoning in large audio–language models for audio question answering (AQA), where a model must infer the correct answer from acoustic evidence and textual answer options. Although reinforcement learning (RL) with verifiable rewards has recently improved reasoning alignment in text and vision tasks, its role in audio remains underexplored because auditory inputs involve temporal structure, perceptual ambiguity, and signal-grounded decision making. To address this gap, we propose a two-stage training framework built on Qwen2.5-Omni-7B. The first stage performs supervised fine-tuning (SFT) on AQA data to establish task alignment and factual grounding from acoustic cues, and the second stage applies Group Relative Policy Optimization (GRPO) with verifiable rewards to directly refine answer correctness and output behavior. On the MMAU Test-mini benchmark, the proposed SFT+GRPO framework achieves the highest average accuracy among the compared methods, reaching 78.87% and outperforming the SFT-only and GRPO-only variants. Further analyses of training stages, RL objectives, reward design, data efficiency, and prompt formats suggest that verifier-guided implicit reasoning is more effective than enforcing long explicit reasoning traces in this AQA setting. These findings suggest that auditory reasoning can benefit from training strategies tailored to audio signals and objective answer verification, and they provide empirical evidence for applying RLVR to large audio–language models.

Read PDF

Similar papers

Listen Then Reason: Perception-Grounded Test-Time Reinforcement Learning for Large Audio-Language Models

Perception-Grounded TTRL (PG-TTRL) is proposed, which aligns label-free test-time optimisation with perceptually grounded reasoning, encouraging the model to structure its reasoning more strongly on the audio input, showing the value of perceptual-grounding optimisation for test-time audio reasoning.

Jia-Heng Dong, Xiao-Feng Yu, Jean Honorio et al. · 0 citations

TTRSD: Test-Time Reinforcement Learning with Self-Distillation for Vision-Language Models

TTRSD separates update direction from update position, determined by group-relative advantages, from update position, determined by visual sensitivity, without requiring ground-truth labels, external verifiers, or a separate teacher, demonstrating cross-dataset generalization while preserving inherent reasoning integri...

Shu-Ning Wang, Zhi-Heng Wu, Xun-Lan Zhou et al. · 2 citations
#machine learning Preprint Oct 2026

AdaLoop: Adaptive-Depth Latent Reasoning for Audio Language Models

Large audio language models answer questions about speech, sound, and music, yet their accuracy drops sharply on tasks that need fine-grained acoustic analysis. Judging which of two speakers has the higher pitch demands iterative signal-level reasoning that a content question does not. Current models spend the same com...

Seung-Woo Lee, Bo-Wen Qi · 0 citations

Reward-Tilted On-Policy Distillation for Acoustic Grounding in Audio-Language Models

Experiments with silenced and replacement audio suggest that RT-OPD strengthens the student's reliance on acoustic evidence, and the 3B model achieves 72.72% accuracy on MMAU, the highest among the compared 3B models and competitive with several 7B and 8B models.

Kai-Yang Li, Shaobo Han, Yue Tian et al. · 0 citations

Mizar: A 159M-Parameter Audio-Language Model for Audio Understanding

Mizar, a 159.3M-parameter ALM, is introduced, a recipe that brings together architecture, data, and three-stage training to build Mizar, a 159.3M-parameter ALM that surpasses the previous best-performing ALM below 200M parameters on all three benchmarks.

Kai-Yang Li, Shaobo Han, Yue Tian et al. · 0 citations
Preprint Sep 2026

AudioICL-Bench: A Benchmark for Large Audio Language Model In-Context Learning

AudioICL-Bench is introduced, a diagnostic benchmark whose per-episode rules are resampled so that no correct answer is recoverable from prior knowledge, and its nine tasks are organized along two axes that separate what must be learned from demonstrations from what must be perceived in the signal, enabling failures to...

Jia-Hung Chen, Yi-Cheng Lin, Kai-Wei Chang et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.