Aug 2026· IEEE Transactions on Software Engineering· Vol 52, pp. 2233-2249· 0 citations· 34 references
Abstract
Generative Artificial Intelligence (GenAI) is increasingly integrated into software products to enable new features and user capabilities, from early exploration to operational deployment. GenAI adoption as a component within a software system introduces quality risks because GenAI outputs are probabilistic, prompt-sensitive, and may drift after release. Organizations, therefore, need to decide what to evaluate, when to evaluate, and who owns quality evaluation activities across software design, development, and operations. ISO/IEC 25059 standard distinguishes between software product quality (e.g., usability) and quality-in-use (e.g., satisfaction) for AI-enabled software, yet it provides limited operational guidance for these evaluation activities. We therefore investigate how industrial software teams adopt and use GenAI models in the software systems they build and operate, and how they evaluate system qualities when deciding to adopt GenAI during development and after deployment. We do not benchmark the underlying GenAI model itself. In this study, we conducted 19 semi-structured interviews in two software development companies. We triangulated the interviews with archival data (15 internal documents and 184 internal wiki/web pages) to capture GenAI adoption steps, quality concerns, evaluation practices, and role responsibilities. Our findings describe a three-phase adoption process – Ideation, Development, and Operation – highlighting where quality evaluations occur, which criteria are used, and how evaluation responsibilities are distributed. Based on observed practices and using ISO/IEC 25059 as an organizing lens, we synthesize a process-oriented quality evaluation framework. This framework maps metrics to explicit gatekeeping, validation, and monitoring checkpoints, bridging abstract ISO quality characteristics with engineering workflows. We applied the framework in a GenAI-enabled software product (SE4AI) use case and reported how it supported structured evaluation activities. We also observed that quality evaluations span legal, security, development, QA, and operations, but ownership is fragmented across phases. We therefore propose a GenAI Quality Lead responsibility (often assignable to an existing senior role) to coordinate criteria, evidence, and traceability across quality evaluation activities. The results contribute to Software Engineering for AI (SE4AI) by clarifying how teams can measure qualities when building software that adopts and uses GenAI.
The goal of this study is to understand practitioners'decision making process and security considerations in selecting and integrating AI components through an exploratory semi-structured interview study, and distill the findings into actionable recommendations for AI adopters, model providers, and researchers.
Mahzabin Tamanna, Elizabeth Lin, Sparsha Gowda et al.· 0 citations
The adoption of artificial intelligence (AI) in organizations is often fuelled by promises of improved efficiency and innovation, while its practical value and operational impact remain unclear. This paper presents a research framework for analysing the added value of generative AI (GenAI) in logistics organizations, with a focus on “difficult to automate” tasks and processes. It combines literature-based model development, semi-structured expert interviews, field studies, and user-cantered experiments to examine how GenAI affects task performance, user behaviour, and organizational structures. It integrates qualitative insights with experimental and modelling approaches to support a systematic assessment of value creation, efficiency gains, usability, user acceptance, and organizational impact. Preliminary results from expert interviews indicate that GenAI is primarily used for cognitive support enabling time savings in manual tasks, and less for process automatization. Eventually, the presented framework will provide a structured basis for the development of evidence-based roadmaps for user-centred AI adoption in logistics.
Gerald Schneikart, Walter Mayrhofer· Engineering review· 0 citations
Technology has revolutionized software development, forcing developers and operations to collaborate to design systems that are resilient, scalable, and efficient. This paper analyzes the two software architectural approaches namely Monolithic Architecture vs Microservices Architecture when considering scalable software development, we put the following into consideration: empirical trade-offs, scalability consideration, organizational implications, and development complexity. Some case studies were considered such as Banking sector, media company and a web-based travel application. In monolithic architecture, the application is constructed using one module only. The user interface, business logic, and data access are combined using the same coding framework. As all modules will be built using one framework, implementation and deployment become easier. Monolithic systems might struggle as applications grow. It is also associated with the following: In terms of cost, it is low, leading to fastest and cheapest way to launch the first product. In the context of skillset, easier for new hires; skills are common. In the aspect of maintenance, high code debt makes maintenance very expensive. In comparison, microservices architecture divides an application into smaller services that carry out specific business tasks. This is accompanied by the following features: From a financial perspective, there are high costs required at the start of implementation due to new tool purchases. Regarding skills, it requires specialists who have experience working on complex systems. When it comes to maintenance, it involves lower costs and targeted updates. In order to address challenges related to scalability and high performance, the team needs to determine the architecture that will work for them to create robust and easily deployable systems.
Theophilus Bamise Ajala, A. Oduroye, I. Ayoade et al.· Asian journal of applied sci...· 0 citations
The increasing availability of Artificial Intelligence (AI) tools has generated significant interest within the project management community; however, structured guidance tailored specifically to Project Managers remains limited. This paper proposes a standardized AI-enabled prompt architecture aligned with the PMBOK® 8E performance domains and processes. The framework consists of a Master Prompt, Task-Level Executable Prompts, and Refine Prompts designed to create bounded, context-aware interactions between Project Managers and AI systems. The proposed architecture embeds PMBOK-aligned terminology and follows the Inputs–Tools–Outputs (ITTO) logic. The AI system functions as an analytical and generative tool within this structure, processing structured and unstructured inputs—including expert judgment—and producing standardized outputs for managerial review and refinement. The architecture is designed to be generalized and extensible across all forty project management processes. This study does not present empirical performance metrics; rather, it introduces a structured conceptual framework intended for practical application and future validation. Practitioners are encouraged to implement the architecture in real project environments to evaluate measurable improvements and contribute to further academic development in AI-enabled project management.
Vittal Anantamula, Rajendra Harsh· 2026 6th International Confe...· 0 citations
Software defects discovered late in the delivery lifecycle remain one of the most expensive and disruptive problems in enterprise software engineering. Test-Driven Development (TDD) and Behaviour-Driven Development (BDD) are two complementary, test-first methodologies that shift quality assurance activity to the earliest stages of development. This paper presents a comparative and architectural study of TDD and BDD, describing their underlying workflows, tooling ecosystems, and the organizational conditions under which each methodology is most effective. We propose an integrated reference architecture that combines TDD's fine-grained unit-level feedback loop with BDD's stakeholder-readable acceptance criteria inside a single continuous integration and continuous delivery (CI/CD) pipeline. The architecture is illustrated with implementation examples in Java using JUnit and Cucumber, and is evaluated against defect-leakage, regression-coverage, and cross-functional-communication criteria drawn from prior empirical studies. The results indicate that organizations combining TDD and BDD achieve lower production defect rates and improved requirement traceability compared to organizations that adopt either practice alone. The paper concludes with practical recommendations for enterprise QA teams planning to adopt a combined test-first strategy.
Urvish Gajjar· International Journal of Sci...· 0 citations
A large-scale empirical study of mainstream open-source agent frameworks from an engineering perspective, providing empirical evidence linking framework design choices to engineering risks and highlighting the need for stronger guidance and support in agent framework development.
Yibo Zhai, Junjun Si, Yan Wang et al.· SIGSOFT FSE Companion· 0 citations