2026· International Conference on Software and Data Technologies· pp. 109-116· 0 citations· 28 references
Computer Science
TL;DR
Investigating LLMs as metric-driven refactoring assistants rather than code generators suggests that while LLMs are valuable assistants for structural improvement, their interventions require careful monitoring to avoid unintended trade-offs.
Abstract
: The growing adoption of large language models (LLMs) in software engineering has introduced new opportunities but also risks in the software maintenance lifecycle. While LLMs can generate entire codebases from natural language prompts, such automatically generated or rapidly prototyped code often accumulates structural debt, making systematic refactoring increasingly urgent. This work investigates LLMs as metric-driven refactoring assistants rather than code generators. Six models (ChatGPT, Claude, Gemini, Grok, DeepSeek, and Qwen) were evaluated on two types of Java projects: three controlled applications with manually inflated structural metrics, and three real-world applications from public GitHub repositories. Using MetricsReloaded in IntelliJ IDEA, we measured four CK metrics: complexity (WMC), cohesion (LCOM), coupling (CBO), and inheritance depth (DIT). Results indicate that LLMs significantly reduce complexity and coupling, improving class simplicity and modularity. However, cohesion improvements remained limited, with LCOM proving especially elusive. Inheritance depth showed strong reductions in synthetic high-metric applications but minimal change in real projects. ChatGPT produced the most consistent and structurally stable refactoring outputs in real applications, though occasional cohesion deterioration occurred. These findings suggest that while LLMs are valuable assistants for structural improvement, their interventions require careful monitoring to avoid unintended trade-offs.
Context: Architectural Design Decisions (ADDs) capture the rationale behind the structure and evolution of software systems but are rarely documented explicitly, and are often hidden inside source code commits. Recovering them is important for Architectural Knowledge Management (AKM). Problem: Extracting ADDs from commits is challenging due to their implicit and unstructured nature. Large Language Models (LLMs) have shown strong capabilities in understanding code and text, yet their effectiveness for this task remains underexplored. Study: We present a preliminary study using four LLMs (Gemini 3 Pro, DeepSeek R1, Kimi K2, Qwen3) with zeroshot and fewshot prompting on 30 developer-written ADDs from open-source projects. We score outputs with ROUGE-L, BLEU, METEOR, and BERTScore, and one author manually reviews the Gemini outputs. Results: All models reach a BERT-F1 above 0.81, and fewshot prompting improves alignment (Gemini BERT-F1: 0.828 to 0.847). However, the generated ADDs are often too long, implementation-focused, and miss the rationale behind the decision. This highlights opportunities for architecture-aware LLM systems and automated AKM.
Amey Karan, Rudra Dhar, Mohamed Soliman et al.· 0 citations
Results show that ML-enhanced recommendations outperform traditional methods in accuracy, relevance, and impact on maintainability metrics, and highlight the potential of integrating ML into modern development practices to support developers in producing cleaner, more maintainable software systems.
Rohit Malhotra· International Journal of Mod...· 0 citations
This work introduces RepoProbe, a novel benchmark for evaluating repository-level code understanding through open-ended Q&A using GitHub Discussions, which focuses on open-ended architectural inquiries rather than defect reporting and proposes a Checklist-Based Verification Protocol that decomposes answers into atomic, verifiable facts, thereby replacing subjective ratings with objective verification.
Recently, Developers have been relying on AI tools to support them in their daily work by generating code. While the use of large language model-based AI tools has improved productivity, the quality of the generated code wasn't always optimal. In a lot of cases, the code includes design issues known as code smells, which negatively impact readability, maintainability, and future development. This paper investigates these issues in AI-generated Java code, with a focus on common object-oriented problems such as switch statements, temporary fields, and refused bequest. A structured approach is proposed that combines static analysis tools with explainable AI techniques to better understand why these problems appear. Based on the realized insights, prompts are optimized to guide the AI model towards generating cleaner and more structured code. The results showed clear improvement after the prompt optimization, where the number of detected code smells was reduced by 66%, and completely removed (100% reduction) in some cases. Overall, the study showed that improving prompt design, supported by explainable analysis, can significantly enhance the quality of AI-generated code.
Y. Younes, Yousef Elsheikh· IEEE Jordan Conference on Ap...· 0 citations
This study compares the structural quality of code produced by three widely adopted vibe coding tools --- Lovable, v0, and Replit --- starting from a single generation prompt and suggests that choosing between vibe coding tools involves structural trade-offs that go beyond perceived productivity.
An automated, multi-dimensional evaluation framework for C# code generation, applying it to four state-of-the-art LLMs: GPT, Gemini, Claude, and Grok is presented and a substantial gap between correctness and quality attributes is revealed.
Seyed Mohammad Mahdi Ghalandarian, Majid Bazargani, Masoumeh Taromirad· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
