A Comparative Analysis of ChatGPT-Generated and Human-Written Use Case Descriptions

The development of comprehensive use case descriptions is a critical task in software engineering, providing essential insights for requirement analysis and system design. The advent of advanced natural language processing models, such as ChatGPT, has sparked interest in their potential to automate tasks traditionally performed by humans, including the generation of use case descriptions in software engineering. Understanding the capabilities and limitations of ChatGPT in generating use case descriptions is crucial for software engineers. Without a clear understanding of its performance, practitioners may either overestimate its utility, leading to reliance on suboptimal drafts, or underestimate its capabilities, missing opportunities to streamline the drafting process. This paper addresses how well ChatGPT performs in generating use case descriptions, evaluating their quality compared to human-written descriptions. To do so, we employ a structured approach using established quality guidelines and the concept of "bad smells" for use case descriptions. Our study presents the first attempt to bridge the knowledge gap by offering a comparative analysis of ChatGPT-generated and human-written use case descriptions. By providing an approach to objectively assess ChatGPT's performance, we highlight its potential and limitations, offering software engineers insights to effectively integrate AI tools into their workflows.