2026· SemEval@ACL· pp. 48-53· 1 citation· 9 references
Computer Science
TL;DR
A systematic ablation comparing five approaches to identify the more narratively similar of two candidate stories relative to an anchor identifies five systematic failure categories, distinct from the All Correct / Partial cases, including 23 Lexical Trap cases, 23 Hard Cases, and 24 Proposed-Recovery cases.
Abstract
This paper presents Team HausaNLP’s sub-mission to SemEval-2026 Task 4 (Track A), which requires identifying the more narratively similar of two candidate stories relative to an anchor. Narrative similarity is defined along three dimensions: abstract theme, course of action, and story outcomes. We conduct a systematic ablation comparing five approaches: a lexical TF-IDF baseline, two bi-encoder SBERT variants ( all-MiniLM-L6-v2 and all-mpnet-base-v2 ), a paraphrase-focused embedding model, and a cross-encoder re-ranker. On the 200-instance development set, all-mpnet-base-v2 achieves the best performance (61.5% accuracy, 61.48 macro-F1), out-performing both TF-IDF (54.5%) and the official SBERT baseline (55.0%). Surprisingly, the cross-encoder re-ranker (55.5%) does not improve on the bi-encoders, which we attribute to the long-document nature of Wikipedia story summaries exceeding the model’s effective context window. On the official test set, our primary SBERT MiniLM submission achieved 61.50% accuracy (33rd of 44 teams). Our error analysis over 200 development instances identifies five systematic failure categories, distinct from the All Correct / Partial cases, including 23 Lexical Trap cases, 23 Hard Cases, and 24 Proposed-Recovery cases, thereby informing concrete directions for future work.
Error analysis shows that a non-trivial fraction of failures are placeholder strings caused by API errors rather than incorrect generations, and that surface-level mismatches (verbosity, ortho-graphic variation) account for many of the remaining errors.
B1ade, an efficient RAG architecture comprising two purpose-built components: a compact embedding model and a purpose-built SLM shows that strategic model composition and reward design suffice for resource-efficient RAG, without large-scale pretraining.
S. Subramanian, M. Gungor, Vikram Elango· arXiv.org· 1 citation
DCASE~2026 Task~5 introduces Audio-Dependent Question Answering (ADQA), which tests whether large audio-language models answer from the audio rather than from textual priors. An Audio-Dependency Filtering (ADF) pipeline combines silent-audio probing, per-option perplexity, a large language model (LLM) commonsense check, and human review to remove items solvable from text alone. The 3000 items that pass form the ADQA-Bench evaluation set, spanning music, speech, and environmental audio. The inaugural edition draws 14 teams and 36 submissions across two tracks defined by total parameter count (up to 100B and under 10B). A Chung-Ang University ensemble of MOSS-Audio-8B-Thinking and Qwen3-Omni-30B reaches the top overall accuracy at \pct{58.33}, and a MOSS-only configuration from the same team leads the sub-10B track at \pct{57.30}. Across the 30 submissions with a comparable development score, evaluation accuracy falls by 11.91 percentage points (pp) on average (median 10.91\,pp) on the hidden evaluation split, which is designed to be harder than the development split. The most common building blocks are: the MOSS-Audio-8B-Thinking backbone (13 of 36 submissions), Low-Rank Adaptation (LoRA) fine-tuning on AudioMCQ-StrongAC, and preference or reinforcement-learning objectives -- Group Relative Policy Optimization (GRPO) in five teams, Group reward-Decoupled Normalization Policy Optimization (GDPO) in two. At test time, prompt engineering is near-universal, and majority or choice-permutation voting is common. Every system misses the same set of 233 evaluation items.
Haolin He, Renhe Sun, Zheqi Dai et al.· 1 citation
SemPOI-RL is proposed, a framework that aligns LLM semantic reasoning with structured sequence generation for interpretable OOT recommendation and consistently outperforms both traditional recommenders and direct LLM baselines, while providing interpretable style attribution across different phases of a trip.
Yunqi Liu, Yang Zhang, Ruixing Zhang et al.· 0 citations
The hugang11 system addresses a practical trade-off in creative text generation: models that produce sharper and more stylized jokes often become less stable in output format, and builds a three-stage pipeline that combines chain-of-thought-augmented supervised fine-tuning (CoT-SFT), teacher-constructed direct preference optimization (DPO), and deterministic post-processing.
