Parth Kulshreshtha

Same Output, Different Gold: Measuring How Reference Choice Moves a Multilingual Benchmark Score

A benchmark score compares a system output against a reference, and methodological attention falls almost entirely on the first term. We measure the second. The retained annotation record of a six-language benchmark for personally identifiable information contains two independent annotator labellings, the aggregate shi...