Jun 2024· arXiv.org· Vol abs/2406.07021· 13 citations· 26 references
Computer Science
TL;DR
A web-based software tool is introduced that employs an LLM-based agent and prompt engineering to automate the generation of test case scenarios against user requirements and crafting test case scenarios based on these stories.
Abstract
Large Language Models (LLMs) are widely used in Software Engineering (SE) for various tasks, including generating code, designing and documenting software, adding code comments, reviewing code, and writing test scripts. However, creating test scripts or automating test cases demands test suite documentation that comprehensively covers functional requirements. Such documentation must enable thorough testing within a constrained scope and timeframe, particularly as requirements and user demands evolve. This article centers on generating user requirements as epics and high-level user stories and crafting test case scenarios based on these stories. It introduces a web-based software tool that employs an LLM-based agent and prompt engineering to automate the generation of test case scenarios against user requirements.
This systematic review presents an analysis of the "Vibe Coding" methodology — a contemporary approach to the iterative software development process using Large Language Models (LLMs). Code generation tools are transforming software development by enabling programmers to formulate tasks and describe the desired behavior of software in natural language, while LLMs generate source code corresponding to these requests. The review systematizes current methodologies for the use of LLMs, highlights application examples, evaluates the effectiveness of generated code, discusses emerging challenges, and outlines future development trends of the technology. The aim of this work is to provide a comprehensive understanding of the capabilities and limitations of Vibe Coding as a transformational methodology in software engineering.
A. Dzhonov, S. M. Avdoshin· INFORMACIONNYE TEHNOLOGII· 0 citations
Software testing plays a fundamental role in the software development lifecycle, because it helps to verify and ensure the quality of the products being built. The results obtained during this stage help to reinforce confidence in the final product while providing valuable insights to support decision-making among project stakeholders. Given the increasing complexity of today’s software systems and the growing demands of clients, organizations must adopt innovative strategies to identify defects early and reduce the risks of failure. This article extends the method originally presented in Design of Test Cases Based on the Language Extended Lexicon and the Requirements Specification. In this extended version, the method has been enhanced by incorporating active participation from both the testing team and the client during the test-case generation stage. Additionally, the third step integrates an AI agent acting as an expert tester in functional testing, collaborating with the testing team to refine and improve the generated test cases.
Julieta Screpnik, L. Antonelli, María Alejandra Paz Menvielle· Cadernos do IME - Série Info...· 0 citations
Code generation aims to automatically generate source code from task requirements and has attracted significant attention with the rapid advancement of large language models (LLMs). Despite remarkable progress, LLMs often struggle to generate correct code for complex software engineering tasks because task descriptions are frequently incomplete, ambiguous, or lack critical contextual information. Existing approaches primarily improve the capabilities of coding agents through more sophisticated tools, skills, and workflows, while largely overlooking the quality of the task requirements themselves. To address this limitation, we draw inspiration from software requirements engineering and propose WiseSpec, a novel requirements-driven agent framework for repository-level code generation. WiseSpec automatically constructs structured and information-rich requirements, assesses their quality through execution-based evaluation, and iteratively refines them to better guide code generation. Experimental results show that WiseSpec consistently outperforms all baselines, achieving an average improvement of 13.17% in %Resolved.
This paper addresses automated unit test generation with large language models (LLMs). LLM-based test generation has not yet attained a quality level sufficient for practical use in industry. Although LLMs often reproduce API syntax faithfully, they frequently disregard semantic usage constraints and execution-environment dependencies, leading to assertion failures, mock-related errors, and reference/resolution errors. A prior failure analysis of Java unit test generation using GPT-4o classified 2980 trials into eight failure patterns and identified three root-cause mechanisms: external context ignorance, internal context ignorance, and a syntax–semantics gap. Building on that analysis, this paper proposes a prompt design comprising three strategies: (1) making the execution state explicit in the generated test, (2) stating semantic constraints explicitly, and (3) injecting environment constraints prior to generation. In contrast to generic techniques such as few-shot learning or chain-of-thought prompting, each proposed strategy is tied to a specific root-cause mechanism, yielding a systematic design in which each rule is explicitly justified by its correspondence to a specific root-cause mechanism. Experiments on 298 methods with five models (GPT-4o, GPT-5, GPT-5.1-Codex, Claude Sonnet 4.5, and Gemini 2.5 Pro) show improved test execution success rates for every model, with absolute gains ranging from 1.1 to 21.1 percentage points (pp). Mock-related errors were reduced by 61.9%–99.2% relative to the baseline prompt, demonstrating effectiveness against the targeted failure patterns. Finally, conditions under which the strategies transfer to other code-generation tasks are discussed, along with limitations on their scope.
This work proposes TraceDev, a multi-agent framework for automated software development grounded in use cases that contain multiple functional points and complex semantics, and demonstrates the effectiveness of TraceDev in repository-level code generation from requirements.
Evaluated on real-world system-level requirements documents, comprising more than 720 requirements and 72 use cases, the approach generates system-level diagrams comparable to those created by experts and provides valuable architectural recommendations.
Bastian Franze, Dominik Fuchß, Friedrich Wattenberg et al.· IEEE International Requireme...· 0 citations
The new ChartNet training dataset could improve the accuracy of vision-language models that help analyze business trends or interpret scientific figures.
