Small Language Models for Pay Equity Compliance: A Fine-Tuning and Evaluation Study
TL;DR
It is asked whether small open-weight language models, fine-tuned and run entirely on local hardware, can do gender pay gaps reporting work without the data leaving the organisation without dedicated machine-learning infrastructure.
Abstract
The EU Pay Transparency Directive (2023/970), applicable from June 2026, requires large employers to report gender pay gaps with documentation of how they were calculated. Turning raw statistical outputs into compliant, readable reports is a barrier for human-resources teams without data-science expertise. Cloud language models could help, but pay data is special-category personal data under the GDPR and risky to send to external services. We therefore ask whether small open-weight language models, fine-tuned and run entirely on local hardware, can do this work without the data leaving the organisation. Using a memory-efficient method (QLoRA, quantized low-rank adaptation), we adapt Llama 3.1 8B, Mistral 7B and Phi-3.5 Mini for three tasks: turning an analyst’s question into statistical parameters, explaining pay-gap results faithfully, and checking a report against the Directive. All run on a mid-range workstation with an 8 GB GPU. Fine-tuning clearly helps the first and third tasks, whereas faithful explanation depends more on retrieving worked examples at inference time. Accuracy saturates at about 40 training examples in total (around 13 per task), indicating feasibility without dedicated machine-learning infrastructure.