A frame-independent audio-text system for the 3rd Ambivalence/Hesitancy Video Recognition Challenge at the 11th Affective&Behavior Analysis in-the-Wild (ABAW) Workshop, ranked third overall on the official challenge leaderboard.
Abstract
We present a frame-independent audio-text system for the 3rd Ambivalence/Hesitancy Video Recognition Challenge at the 11th Affective&Behavior Analysis in-the-Wild (ABAW) Workshop. Videos are divided into overlapping 5-s windows aligned with transcript timestamps. Each window combines prosodic audio descriptors, emotion-oriented RoBERTa embeddings, and 74 psycholinguistic features representing uncertainty, hedging, and attitudinal conflict. Temporal cross-attention fuses audio and text, while the support features condition gated Multiple Instance Learning (MIL) pooling. A five-seed ensemble achieves an average precision of 0.875 and a macro-F1 of 0.722 on the 525-video labeled public-test split. Notably, our submission ranked third overall on the official challenge leaderboard, with a macro-F1 of 0.7455. Source code is available at https://github.com/Liga-de-IA-PUCPR/abaw-11-ah-challenge/.
This work reformulates AER as a Temporal Affective Grounding task that associates emotions with temporally bounded speech spans and vocal tone descriptions, and introduces Masked Temporal Affective Grounding (M-TAG), a supervised training objective that combines full-sequence language modeling with emotion and timestam...
Abdelrahman Mohamed, Lars Kai Hansen, Zheng-Hua Tan· 0 citations
The results show that the evaluated baselines can extract some affective signal from speech, but discrete expressed-emotion recognition remains fragile, especially for non-neutral emotions that are often most important in voice agent workflows.
Luc Debaupte, Tyler Baumgartner, Brandon Tai et al.· 1 citation
An exploratory weakly supervised NLP analysis of positive/negative affective-tone proxies in AphasiaBank transcripts with paired audio highlights both the promise of clinical NLP for aphasia discourse analysis and the need for independent human-labeled validation, utterance-aligned acoustic features, and careful contro...
Shamiha Binta Manir, A. Kothari, William L. Gross et al.· Frontiers in Digital Health· 0 citations
Fake-speech detection is commonly studied through acoustic artifacts, speaker-level spoofing cues, or visual inconsistencies, while less attention has been given to meaning-level manipulation, where the spoken content is altered while the speech remains natural and speaker-consistent. This study investigates the relati...
Alaa Alsaeedi, Amal Almansour, A. Jamal· Applied Sciences· 0 citations
Speech emotion recognition (SER) is the task of assigning emotion labels to utterances. Early systems relied on acoustic features, whereas recent approaches combine multiple modalities, most commonly speech and text. Still, performance remains poor on many datasets. Large language models (LLMs) have therefore attracted...
Detecting true felt emotions when speakers suppress or mask their internal state poses a fundamental challenge for affective computing systems. We study a specific form of affective dissonance in dyadic speech: utterances where a speaker’s self-reported emotion diverges from all external observer ratings, indicating th...
Cheng-Shiuan Lin, T. Eze, Da-Wei Xie et al.· Proceedings of the 28th Inte...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
