SankeyQA, career-map Sankey diagrams with exact answers, benchmark data, model replies and code

Abstract

SankeyQA is an open benchmark for testing whether vision-language models can answer questions about career-map Sankey diagrams, of the kind shown by the XR-CareerAssist virtual reality career guidance platform (VOXReality, Horizon Europe). The data are artificial. All 600 career cohorts, their flow tables and the 1,260 career-map images were generated by a computer program only to demonstrate and test the technology. They are not real careers, contain no personal data and do not come from any real career database, and no conclusion about real roles, industries or people should be drawn from them. The deposit contains the flow tables, 1,260 images in twelve drawing styles, 6,560 questions whose answers are computed exactly from the flow tables, the code that generates the benchmark, evaluates five small open models (SmolVLM-256M and 500M, MatCha-ChartQA, BLIP-VQA, Qwen2.5-0.5B) and two baselines, and fine-tunes BLIP-VQA on 20 to 400 maps, every model reply, all result tables, the figures (EPS, PDF, 600 dpi TIFF, 300 dpi PNG) and the build scripts of the accompanying article, "Reading the picture or reading the data? An open benchmark of career-map Sankey diagrams for small vision-language models" (submitted to Open Research Europe). Files. sankeyqa-bench-v1.0.0_code_data_results.zip holds the complete repository except the images, which are in sankeyqa-bench-v1.0.0_images_part01.zip to part19.zip (unzip all into the repository folder). The images can also be regenerated byte for byte with code/scripts/make_dataset.py. See README_ZENODO.md. Source code also on GitHub, https://github.com/ntantaroudas/sankeyqa-bench. Licences. Code under the Apache License 2.0, data, model replies, results and figures under CC BY 4.0. Funded by Financial Support to Third Parties of the VOXReality project (Horizon Europe, Grant Agreement No. 101070521).