Generative AI for Plain Language: A Comparative Study of Prompt Strategies and Linguists Validation.
TL;DR
It is concluded that adopting LLMs in the public sector demands “Human-in-the-loop” workflows to ensure legal accuracy and clear communication.
Abstract
The technical complexity and “legalese” inherent in official documents create significant cognitive barriers that reduce public transparency and hinder the effective exercise of digital citizenship. While manual plain language and Natural Language Processing (NLP) initiatives exist in the public sector to mitigate these problems, scaling these efforts to handle massive volumes of text remains a significant challenge, highlighting the need for automated solutions. In this context, the main objective of this study is to evaluate the use of Generative Artificial Intelligence (GenAI) as a tool for rewriting official documents according to plain language guidelines. We generated 18 versions of six document typologies using GPT-4o-mini, comparing zero-shot, one-shot, and chain-of-thought prompting. The evaluation employed a hybrid approach: qualitative assessment by expert linguists alongside statistical readability (Flesch-Kincaid) and semantic similarity metrics (BERTScore, BLEU). Additionally, the “LLM-as-a-judge” pipeline was implemented to test automated reviewing. Results show that while GenAI consistently improves readability to high-school levels, no single prompt strategy is universally optimal, and its effectiveness heavily depends on the document’s structure. Furthermore, the LLM-as-a-judge experiment failed to replicate human judgment, showing low correlation with experts. We conclude that adopting LLMs in the public sector demands “Human-in-the-loop” workflows to ensure legal accuracy and clear communication.
