Jul 2026
What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks
Official short-answer VQA scores should be accompanied by semantic audits and answer-type diagnostics to remain interpretable, because instability is structured by answer type: extractive and multi-span answers are far more evaluator-sensitive than scalar answers.
Guanhua Ye, Jing-Bin Niu, Yan Li et al.
· arXiv.org · 0 citations