Data, code and model outputs for: Benchmarking large language model elicitation formats against expert analytic hierarchy process weights for soil quality mapping in the Tigris River Basin

Abstract

Version 2.0.0. This version archives the elicitation experiment repeated on 20 September 2026 with exactly the 11 indicators, 55 pairwise comparisons and English prompts described in the manuscript, and every analysis recomputed from it. It supersedes version 1.0.0, whose archived run had presented 13 parameters and 78 comparisons in Turkish. The input data, the expert reference and the citation audit carry over from version 1.0.0 unchanged; README.md states what changed and why. Reproducibility package for the manuscript Benchmarking large language model elicitation formats against expert analytic hierarchy process weights for soil quality mapping in the Tigris River Basin (Kilic, Gunal and Budak), submitted to Computers and Electronics in Agriculture. What the study did. The same 55 pairwise comparisons among 11 soil quality indicators were put to four large language models under two elicitation formats: all 55 comparisons inside one prompt (Scenario A, joint), and one comparison per prompt with no access to the others (Scenario B, independent). Analytic hierarchy process weights from four soil scientists provide the human reference. The resulting weight vectors were applied to 452 soil samples (0-20 cm) from a section of the Tigris River Basin, Turkiye, and the effect on soil quality classification and on mapped class area was measured. Models. Four endpoints called through OpenRouter, pinned to date-stamped identifiers: openai/gpt-5.6-luna-20260709, anthropic/claude-sonnet-5-20260630, x-ai/grok-4.3-20260430, and the open weights model nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16. Extended reasoning was disabled on every model. The open weights model is the one component of the panel that a third party can rerun after the commercial endpoints are withdrawn. Contents. 01_input_data.zip - soil indicator measurements, the cleaned analysis table and the scored matrix. 02_expert_reference.zip - derived expert AHP weights, matrix metrics and between-expert distances. Panel members are identified only as U1 to U4. 03_llm_elicitation.zip - raw model responses (5600 records in the main run), parsed judgments (5600 records), validity and reciprocity audits, frozen prompt hashes, the temperature-matched analysis set of 360 matrices used in the manuscript, plus the criterion-arm test, the endpoint precheck and the pilot run. 04_rationale_leakage.zip - the rationale substudy and the citation audit used as a pretraining leakage control. 05_downstream_mapping.zip - soil quality index values, class transition matrices, kriging results, changed area, bootstrap intervals, spatial autocorrelation and hotspot output, including the CR-filtered repeat. 06_synthesis.zip - expert versus model comparison at matrix, vector and map level. 07_code.zip - the analysis scripts in execution order, the experiment configuration and the pinned dependencies. README.md - full file guide, field glossary and instructions for reproducing the analysis. MANIFEST.csv lists every file with its size and SHA-256 checksum. Reproducibility. Every downstream step runs from the archived responses, so the analysis reproduces end to end without a single API call. Model outputs are not deterministic and the commercial endpoints will be withdrawn, which is why the raw responses are archived here rather than regenerated. Not included. The API key, by design. The filled expert spreadsheets, because the four soil scientists recorded their judgments in a third party AHP Excel template that carries its own licence; the derived quantities in 02_expert_reference.zip reproduce every expert value used in the article. The article's rendered figures. The virtual environment, which is pinned instead by requirements.txt and uv.lock. Licence. Data, documentation and outputs under CC BY 4.0. Code in 07_code.zip under the MIT Licence. Note on field language. Column headers and field names are in Turkish, the working language of the analysis. README.md carries a glossary.