On Converting Natural Language Requirements into Semi-Formal Templates Using LLMs

Abstract

Context and Motivation] Semi-formal syntax templates for natural language requirements positively impact various requirements metrics, such as singularity. Using templates such as MASTeR or EARS also improves understandability. Requirements that conform to templates are easier for humans to understand than unrestricted requirements. [Question/Problem] However, converting requirements into templates is time-consuming and requires substantial prior training and in-depth domain knowledge. Thus, most requirements are still written in unrestricted natural language. [Principal Ideas and Results] Our approach is to use large language models (LLMs) to convert unrestricted natural language requirements into templates. Our evaluation demonstrates the proficiency of LLMbased systems. LLM-converted requirements are rated similarly to human rephrasings, especially for shorter requirements. Thus, they can be used to accelerate the requirements rephrasing process. [Contribution] In this paper, we present an approach for automatically parsing free-text requirements into templates with LLMs. We also provide an overview of metrics for automatically validating such systems and use them to compare our rephrased requirements with a ground truth. In a practitioner survey comparing LLM- and human-converted requirements, we assess the validity of these metrics.