Results corroborate the effectiveness of LLMs in improving and streamlining RE phases by analyzing the semantic similarity and API performance of different models, as well as their effectiveness and efficiency in requirements analysis.
Abstract
Requirements Engineering (RE) plays a pivotal role in software development, encompassing tasks such as requirements elicitation, analysis, specification, and change management. Despite its critical importance, RE faces challenges including communication complexities, early-stage uncertainties, and accurate resource estimation. This study empirically investigates the effectiveness of utilizing Large Language Models (LLMs) to automate requirements analysis tasks. We implemented a multi-agent system that deploys AI models as agents to generate user stories from initial requirements, assess and improve their quality, and prioritize them using a selected technique. In our implementation, we deployed four models, namely GPT-3.5, GPT-4 Omni, LLaMA3-70, and Mixtral-8B, and conducted experiments to analyze requirements on four real-world projects. We evaluated the results by analyzing the semantic similarity and API performance of different models, as well as their effectiveness and efficiency in requirements analysis, gathering users' feedback on their experiences. Preliminary results indicate notable variations in task completion among the models. Mixtral-8B provided the quickest responses, while GPT-3.5 performed exceptionally well when processing complex user stories with a higher similarity score, demonstrating its capability in deriving accurate user stories from project descriptions. Feedback and suggestions from the four project members further corroborate the effectiveness of LLMs in improving and streamlining RE phases.
A comprehensive overview of the existing tools and frameworks for implementing MAS in software engineering and a set of lessons learned and challenges that can help researchers and practitioners to select a suitable MAS framework according to their needs are provided.
Maria Sâmyla Serafim de Oliveira, M. Ibiyo, Marco Gianrusso et al.· 0 citations
Context: Software engineering is moving from AI-assisted practices like vibe coding, in which assistants accelerate individual developers, towards Agentic Software Engineering (ASE), in which autonomous agents are delegated goal-level tasks. However, industry reports a productivity paradox: as individual productivity increases, team throughput, review capacity, and stability degrade because team-scale software engineering discipline is neglected. Objective: This paper aims to establish the conceptual and methodological foundations of Spec-Driven Development (SDD) as an enabling discipline for ASE at team scale and characterize the harness, i.e., the technical and methodological mechanisms through which teams govern agent behavior. Method: We conducted a conceptual analysis drawing predominantly on gray literature, including ASE vision and roadmap papers, practitioner reports, talks, and tooling, because peer-reviewed evidence and a shared academic-industrial vocabulary are not yet established. Results: Using a comparative characterization of the paradigm progression as conceptual framing, the article presents (i) a socio-technical model of SDD in which specifications act as the contract substrate between humans and agents; (ii) an operational characterization of the harness, distinguishing the technical harness around the agent from the methodological harness around the team, with worked examples; and (iii) a typology of five human--agent interaction patterns through which the human role is redefined. Conclusion: We conclude that SDD reconstitutes, in specification-centric form, the contracts that vibe coding dissolves: accountability, verifiability, and transferability. Given the immaturity of the evidence base, this work is presented as a first step toward academic-industrial consensus rather than a validated theory, and outlines a research agenda for future empirical validation.
J. Díaz, J. Gayoso, Andrea Cimminio et al.· 0 citations
TestAgent is proposed, an LLM-based test generation approach that addresses the above limitations by emulating human testing practices via a multi-agent collaboration mechanism and equips TestAgent with a set of tool APIs that can be invoked dynamically in an on-demand and adaptive manner.
Quanjun Zhang, Ye Shang, Siqi Gu et al.· arXiv.org· 0 citations
The growing complexity of software systems has increased demand for intelligent solutions in Software Quality Assurance (SQA). Agentic Artificial Intelligence (AI) offers autonomous decision-making, adaptive behavior, and proactive quality improvement, yet its applications across SQA remain fragmented. This systematic mapping study examines Agentic AI integration in SQA, analyzing application distribution, agent characteristics, and supported stakeholder roles to identify coverage, trends, benefits, limitations, and gaps. Following systematic mapping protocols and PRISMA, five digital libraries were searched, retrieving 37 primary studies. Data were classified through frequency analysis and qualitative categorization aligned with Agentic AI behavior, architecture, and autonomy dimensions. Applications concentrate in Product Assurance, mainly test generation and defect management, with fewer studies in Process or Planning activities. Most agents exhibit goal-oriented or diagnostic behaviors, moderate autonomy, and hybrid or multi-agent architectures. Benefits include efficiency, accuracy, and maintainability improvements; limitations involve scalability, generalizability, and transparency. Role coverage focuses on execution-level stakeholders, while managerial and leadership roles receive minimal support. Agentic AI shows potential for strengthening SQA intelligence and adaptability but remains unevenly distributed across activities and roles. Transparency, validation, and leadership support are priority research gaps for developing trustworthy, process-aware Agentic systems across the full SQA lifecycle.
A systematic literature review of technical approaches, including agent architecture, perception, memory, reasoning and planning, action space, orchestration, and self-improvement, reveals a field that has built agents able to act but not yet agents whose authority is bounded or whose behavior is auditable.
Jing-Jing Nie, Jiawei Guo, Krishna Meda et al.· 0 citations
The integration of large language models (LLMs) into software development workflows has given rise to a paradigm known as Agentic Software Engineering (SE 3.0), in which autonomous agents manage full development life cycles under human supervision. This paper presents an exploratory case study in which three LLM-based tools, Antigravity (an agentic IDE with a Gemini 2.5 backend), Gemini CLI, and Qwen Code (local execution via Ollama), are coordinated according to a role-distribution framework proposed in this work as the Role Specialization Model (RSM). Three research questions guide the study: (RQ1) how can LLM-based tools with distinct capabilities be coordinated through the RSM in a real development workflow; (RQ2) what deviations from the planned role distribution emerge during RSM execution and what factors explain them; and (RQ3) how does the resulting product compare against the ISO/IEC 25010 quality model. The objective was to propose the RSM with the incremental development of a Python desktop application for interactive climate-data visualization. The workflow, observed deviations, prompt-hardening techniques, and a qualitative quality assessment are documented. Results suggest that explicit role coordination can support development cycle organization and architectural quality, but requires deliberate coordination strategies, context management, and human verification of agent-generated outputs.
C. Fernández-Y-Fernández, Jorge R. Aguilar-Cisneros· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduJun 3, 2026
The new ChartNet training dataset could improve the accuracy of vision-language models that help analyze business trends or interpret scientific figures.
A new method for surgically removing training examples from a model reveals that as datasets grow, the link between what a model learns and what it produces dissolves.
Microsoft Research Blog· microsoft.comJul 30, 2026
Computer-use AI agents struggle with multi-step workflows like email and customer support. Echoverse trains agents in realistic environments rather than simply providing more training tasks, helping them improve as the tasks, tests, and environments evolve. The post Echoverse: Deep, evolving environments for computer-use agents appeared first on Microsoft Research.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
