This study investigates whether an LLM can bridge requirements engineering and architectural design, generating architectures solely from textual requirements and evaluating structural agreement and perceived quality of results, and shows potential for requirements-driven synthesis when guided by exemplar prompting.
Abstract
Microservice architectures have become dominant for modernizing monolithic systems, yet identifying appropriate services remains challenging and largely manual. Existing decomposition approaches are predominantly code-centric, limiting applicability in early design stages where only textual requirements are available. Despite advances in Large Language Models (LLMs), limited empirical evidence exists on their ability to synthesize complete microservice architectures from natural-language requirements, including service definitions and inter-service interactions. This study investigates whether an LLM can bridge requirements engineering and architectural design, generating architectures solely from textual requirements and evaluating structural agreement and perceived quality of results. We conduct a mixed-method study using OpenAI o3 under zero-shot (ZS) and few-shot (FS) prompting across two systems (Bookstore, PetClinic), one execution per system/condition. Architectures are evaluated through (i) comparison with reference architectures using precision, recall, and F1-score for service identification and communication recovery, and (ii) a blinded expert assessment of correctness, completeness, modularity, and plausibility, plus open feedback synthesis. OpenAI o3 identifies services with higher agreement under FS prompting (F1 = 0.79 for ZS versus = 0.97 for FS). Communication recovery is more challenging: ZS produces dense architectures with high recall but low precision (F1 = 0.61), while FS improves agreement, reaching F1 = 0.82 and reducing unsupported dependencies. Expert evaluation corroborates these results, with FS architectures perceived as more modular, coherent, and plausible than ZS outputs. OpenAI o3 shows potential for requirements-driven synthesis when guided by exemplar prompting. Results are model- and context-specific from two small systems, not model-independent proof.
The findings demonstrate that structural evaluations of LLM-generated decompositions should explicitly control for mapping coverage, as apparent differences between prompting strategies may otherwise reflect methodological bias rather than genuine architectural quality.
D. Silva, Renan Alves, Emanuel Dantas et al.· arXiv.org· 0 citations
A large empirical study on using LLMs to generate CI configurations from natural language across services and model families suggests that similarity and validity are distinct objectives for CI generation and motivate schema-aware evaluation and tooling for LLM-based configuration generation.
Context: Architectural Design Decisions (ADDs) capture the rationale behind the structure and evolution of software systems but are rarely documented explicitly, and are often hidden inside source code commits. Recovering them is important for Architectural Knowledge Management (AKM). Problem: Extracting ADDs from commits is challenging due to their implicit and unstructured nature. Large Language Models (LLMs) have shown strong capabilities in understanding code and text, yet their effectiveness for this task remains underexplored. Study: We present a preliminary study using four LLMs (Gemini 3 Pro, DeepSeek R1, Kimi K2, Qwen3) with zeroshot and fewshot prompting on 30 developer-written ADDs from open-source projects. We score outputs with ROUGE-L, BLEU, METEOR, and BERTScore, and one author manually reviews the Gemini outputs. Results: All models reach a BERT-F1 above 0.81, and fewshot prompting improves alignment (Gemini BERT-F1: 0.828 to 0.847). However, the generated ADDs are often too long, implementation-focused, and miss the rationale behind the decision. This highlights opportunities for architecture-aware LLM systems and automated AKM.
Amey Karan, Rudra Dhar, Mohamed Soliman et al.· 0 citations
Designing Enterprise Architecture (EA) artifacts, particularly during the initiation phase such as the TOGAF Preliminary Phase, is a crucial foundation for business and technology alignment. However, this formulation process often creates an operational bottleneck as it demands high precision, involves heavy administrative documentation, and consumes a significant amount of time. This single-case study aims to evaluate the effectiveness of utilizing a Large Language Model (LLM)—specifically Gemini 3.1 Pro with a structured Prompt Engineering technique—as a solution to accelerate the design of these documents. Through a case study on the development of Smart Estate Management capabilities within the PropTech ecosystem of PT XYZ, the AI-generated EA artifacts were evaluated by five senior IT experts. The evaluation methodology employed an Expert Judgement approach (Likert scale) cross-validated with in-depth qualitative interviews. The findings indicate a strong agreement among the participating experts that the LLM output possesses a high degree of clarity and structural consistency (median score 5.00/5.00). Furthermore, stakeholders perceived that the validated use of AI could provide an estimated time savings of approximately 80% for initial drafting. Nevertheless, the practicality of document implementation was rated neutral (median score 3.00/5.00) because the AI outputs were found to be inherently generic and unable to accommodate tacit knowledge, legacy infrastructure, and company-specific compliance constraints. Ultimately, this research affirms the necessity of a human-in-the-loop paradigm in the EA discipline within this specific context. It suggests that LLMs function effectively as augmentation accelerators rather than human replacements, thereby shifting the enterprise architect's focus from manual drafting to analytical validation and strategic contextualization.
R. Darmawan, Alfa Yohannis· Proceeding of International...· 0 citations
Software Product Lines (SPLs) promise substantial efficiency gains by systematically managing variability across families of related software systems. Realizing these benefits requires a configuration process, performed at compile time, runtime, or both, in which users select and configure features to derive a concrete product instance from an SPL. However, bridging the gap between such feature models and usable product configurations remains a persistent challenge. Users must navigate large variability models, interpret technical terminology, and manually satisfy complex dependency constraints, resulting in significant cognitive load for domain experts and increased development effort. To address these challenges, we draw on principles from Low-Code Development (LCD) and introduce an LLM-driven approach for automatically generating interactive Web-based configurators directly from feature models. Our goal is not only to simplify the development of configurators, but also to support feature selection itself through low-code techniques that make variability accessible to non-technical users. Rather than producing a single interface, our approach synthesizes multiple configuration experiences, including step-by-step wizards, visual dashboards, and profile-based guidance, tailored to different user preferences and mental models. All generated configurators preserve the semantics of the underlying feature model by enforcing variability constraints, while reducing complexity through contextual explanations, intuitive labeling, and guided interactions. We demonstrate the benefits of our approach through exploratory experiments across diverse proxy scenarios and expert interviews with specialists in SPLs and Human-Computer Interaction (HCI).
Patrick Wegerer, Bernhard Schenkenfelder, Rudolf Ramler et al.· 0 citations
An initial characterization of LLM-supported design in self-adaptive systems in SASS is contributed, research directions are outlined, and discussion within the community on advancing LLM-supported architectural design for self-adaptive and autonomous software systems is stimulated.
Nadeem Abbas, Nazia Shahzadi· WiPiEC Journal - Works in Pr...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
