LLM-based multi-agent systems can support strategic AI planning by enabling iterative refinement with human experts by supporting structured and collaborative Requirements Engineering processes for AI adoption planning.
Abstract
Context: Organizations adopting Artificial Intelligence (AI) face challenges in eliciting and analyzing requirements that align with strategic objectives, especially when human oversight and iterative refinement are needed. Large Language Models (LLMs)-based Multi-agent systems provide a potential solution by supporting structured and collaborative Requirements Engineering (RE) processes for AI adoption planning.
Objective: The objective of this study is to investigate whether a multi-agent system, built on LLMs and supported by human input, can assist in requirements analysis for AI adoption. Method: We used a mixed-method approach: (i) designed and developed a multi-agent system to support the generation and prioritization of requirements for AI adoption, (ii) conducted multiple case studies with four companies to evaluate the system, and (iii) collected data through post-session questionnaires from nine participants and follow-up interviews, one per company.
Results: Questionnaire and interview findings together indicate that the system may assist in identifying relevant and goal-aligned requirements. Seven participants considered the generated requirements relevant, and six found them aligned with organizational goals. Participants noted that iterative feedback improved completeness and feasibility, often within two feedback rounds. Both data sources show that human input was essential to clarify technical details, ensure contextual accuracy, and validate prioritization results. Participants from all companies also identified usability, transparency, and scalability as areas requiring further refinement for broader organizational use.
Conclusions: LLM-based multi-agent systems can support strategic AI planning by enabling iterative refinement with human experts. Future work will include more interviews with stakeholders and adjustments to system features to improve transparency, usability, and scalability.
The study argues that collaborative intelligence should be viewed as an organizational capability rather than merely a technological outcome, requiring deliberate management of human judgment, ethical responsibility, and organizational design.
M. R· International Journal of Phi...· 0 citations
Introducing artificial intelligence (AI) teammates into organizations and enabling human–AI collaboration can improve responses to complex problems. However, current methods for identifying AI teammate requirements in complex problem‐solving situations often overlook the experience and knowledge of business staff and lack a human‐centred perspective. Therefore, this study proposes a method for identifying AI teammate requirements for business staff (IATReBS), with a particular emphasis on leveraging their experience and knowledge. We use the design science research methodology to combine theoretical insights from participatory design with practical insights from user interviews (15 participants) to develop IATReBS. The method guides business staff in identifying the requirements of AI teammates for complex problems in specific business contexts. Results from the proof‐of‐concept (15 interviews) and proof‐of‐value (a two‐month experiment yielding 24 questionnaires) evaluations provide initial evidence that the IATReBS method can help participants identify AI teammate requirements in complex problem‐solving contexts, particularly regarding feasibility, usability and perceived usefulness. Our research provides new insights and a method for helping organizations introduce AI teammates from a human‐centred perspective, thereby contributing to the externalization of tacit knowledge. Our research offers methodological guidance for organizations to acquire AI teammates that are aligned with clear application scenarios and meet user expectations.
Wen-Qiang Li, J. Gou, L. Camarinha-Matos et al.· Systems research and behavio...· 0 citations
This paper develops a multi-agent enterprise artificial intelligence (AI) operating model for organizations seeking to move beyond isolated AI assistants toward coordinated AI workforces, positioned within Saudi Vision 2030 and the Kingdom’s national data-and-AI strategy. Using design science research, the study specifies the Saudi Enterprise Multi-Agent AI Operating Model (SEMAI) as a conceptual artifact synthesized from enterprise AI, multi-agent systems, digital transformation, responsible AI, and information systems design science literature. The artifact is demonstrated and formatively evaluated through a structured scenario walkthrough based on a generalized Saudi enterprise context; the study does not claim empirical validation or measured deployment outcomes. SEMAI comprises five layers integrating human roles, AI assistants, collaborative agents, enterprise systems, and governance controls. It is accompanied by design requirements, design principles, an artifact specification, an evaluation rubric, a maturity model with progression criteria, governance and accountability controls, and a vendor-neutral model with a Microsoft-oriented reference implementation. The contribution is a reusable, governance-aware operating model for transitioning from task-level AI assistants to accountable, human-supervised, multi-agent AI workforces suitable for digitally mature Vision 2030 organizations.
The study concludes that Explainable AI is not only a technological enhancement but also a strategic tool for promoting employee trust and supporting effective digital transformation and recommends that organizations prioritize explainability, invest in AI literacy and training, and develop transparent AI governance frameworks to encourage successful adoption of AI-driven business process automation.
Communication inflexibility, limited shared understanding, and trust miscalibration emerge as recurring barriers to HAT, while regulatory capacities represent particularly critical dimensions of HAT readiness that remain to be fully operationalized.
Sébastien Tremblay, Delphine de Hemptinne, Gabrielle Teyssier-Roberge et al.· Human Factors· 0 citations
A comprehensive overview of the existing tools and frameworks for implementing MAS in software engineering and a set of lessons learned and challenges that can help researchers and practitioners to select a suitable MAS framework according to their needs are provided.
Maria Sâmyla Serafim de Oliveira, M. Ibiyo, Marco Gianrusso et al.· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduJun 3, 2026
The new ChartNet training dataset could improve the accuracy of vision-language models that help analyze business trends or interpret scientific figures.
A new method for surgically removing training examples from a model reveals that as datasets grow, the link between what a model learns and what it produces dissolves.
