Xia Li, Kennesaw State University, USA

In modern software development, Non-Functional Requirements (NFR) are essential to satisfy users' needs, which define various constraints and qualities that the system must adhere (e.g., quality, usability, security). Since NFR play a critical role in the guidance of architectural design, it is important to extract different NFR from software requirements specification documents early and accurately. However, distinguishing different categories of NFR is tedious, error-prone, and time-consuming due to the complexity of software systems. In our paper, we conducted a comprehensive study to evaluate the performance of prompt-based non-functional requirements classification by designing various handcraft templates and soft templates on the pre-trained language model (i.e., BERT). Our experimental results show that handcraft templates can achieve best effectiveness (e.g., 83.52% in terms of F1 score) but with unstable performance for different templates. Also, the performance can become stable after soft templates are concatenated with handcraft templates. For example, the standard deviation of F1 score for four combined templates can be improved to 0.74 from 1.00 for handcraft templates.

Non-functional requirements classification, prompt-based learning, pre-trained models.