OSDG-CD Multi-Label SDG Classification Benchmark (5K Sample): LLM and Keyword-Based Predictions
Abstract
Version 2.0.0. Adds five additional LLM classifiers on top of the original v1 release. Predictions for the five newly added models were generated using a more restrictive prompt than the one used in v1; predictions for the original five v1 LLMs are unchanged. See the version history in README.md. A 5,000-entry benchmark subset of the OSDG Community Dataset (OSDG-CD), drawn as a uniform random sample with no filtering, augmented with multi-label Sustainable Development Goal (SDG) classifications from ten state-of-the-art Large Language Models and two keyword-based classifiers. Unlike the original OSDG-CD, which assigns a single SDG label per excerpt, this benchmark provides multi-label predictions: each classifier may assign zero, one, or several SDG labels (1–17) to a given text. This supports evaluation of multi-label agreement, coverage, and overlap between LLM-based and keyword-based approaches. LLM predictions (multi-label, integer SDGs 1–17) Included since v1 — generated with the v1 prompt: deepseek/deepseek-v3.2, anthropic/claude-sonnet-4.6, google/gemini-3-flash-preview, openai/gpt-5.4, qwen/qwen3.6-plus. Added in v2 — generated with a more restrictive prompt: anthropic/claude-opus-5, google/gemini-3.7-flash, openai/gpt-5.6-luna, qwen/qwen3.8-2.4t-a95b, z-ai/glm-5.3-flash. Because the two groups were prompted differently, differences in label counts or coverage between a v1 model and a v2 model partly reflect the prompt change rather than model capability alone. Keyword-based predictions elsevier_2025 — Elsevier 2025 SDG mapping. partaourides2023thematic — consolidated thematic keyword set synthesizing the Aurora, Auckland, and Elsevier families. Keyword-based columns are unchanged from v1. Provenance Derivative of the OSDG Community Dataset (OSDG-CD) v2022.10, released under CC BY 4.0: OSDG, UNDP IICPSD SDG AI Lab, & PPMI (2022), https://doi.org/10.5281/zenodo.7136826. All original OSDG-CD columns are preserved verbatim. Citation Please cite both this dataset and the original OSDG-CD. Cite v2 when using any of the five newly added LLM columns; v1 remains citable via its own version DOI for the original five-LLM set. See README.md for the full schema and details.