Requirement Classification and Prioritization Using Artificial Intelligence and Natural Language Processing
TL;DR
The results indicate that transformer-based representations offer a modest but consistent improvement in contextual understanding over TF-IDF, while classical classifiers remain competitive and considerably cheaper to train.
Abstract
Requirement engineering is a foundational stage of the software development life cycle, and the accuracy with which requirements are classified directly influences downstream design, testing, and cost estimation. Software requirements are commonly separated into Functional Requirements (FR), which describe what a system must do, and Non-Functional Requirements (NFR), which describe how well the system must do it, covering attributes such as performance, security, and usability. When this separation is carried out manually, the process is slow, subjective, and prone to disagreement between analysts, particularly as project size grows. This paper presents a Requirement Classification and Prioritization Tool that combines Natural Language Processing (NLP) with machine learning to automate the FR/NFR decision. Requirement statements are cleaned and normalised, then represented numerically through two complementary techniques: Term FrequencyInverse Document Frequency (TF-IDF) and contextual BERT embeddings. Four classifiers-Logistic Regression, Support Vector Machine, Random Forest, and a BERT-based model-are trained and benchmarked on a dataset of 6,086 labelled requirement statements using accuracy, precision, recall, and F1-score. A Gradio-based interface allows a requirement to be submitted and its predicted category, confidence score, and a short explanation to be viewed immediately. The results indicate that transformer-based representations offer a modest but consistent improvement in contextual understanding over TF-IDF, while classical classifiers remain competitive and considerably cheaper to train.