TestAgent, a multi-agent tool implemented as a VS Code extension that automates the generation of high-quality unit tests for Java projects using repository-level Code Knowledge Graphs, demonstrates its practical utility for regression testing and bug discovery.
Abstract
Automated unit test generation plays a critical role in maintaining software quality, yet existing LLM-based tools often struggle with limited repository-level context and rigid generation workflows. In this paper, we present TestAgent, a multi-agent tool implemented as a VS Code extension that automates the generation of high-quality unit tests for Java projects using repository-level Code Knowledge Graphs. Unlike existing tools that rely on coarse-grained workflows, TestAgent mirrors the structured practices of human developers, who typically analyze requirements before writing tests and review results afterward, by coordinating three specialized agents: a Planner to derive testing requirements, a Generator to synthesize executable code via tool-assisted retrieval, and a Reviewer to iteratively refine tests based on execution feedback. Evaluation on 1,451 methods across six Java projects demonstrates that TestAgent achieves a 92.34% line coverage, significantly outperforming baselines like EvoSuite and ChatUniTest. Furthermore, the tool successfully detects 154 real-world bugs with high precision (92.22%), demonstrating its practical utility for regression testing and bug discovery. The tool source code and demonstration are available at: https://github.com/iSEngLab/TestAgent-VSCode-Extension.
Recently, the emergence of Large Language Models (LLMs) has spurred a surge of research into automated unit test generation, yielding impressive performance and reducing manual effort. However, existing LLM-based approaches still suffer from two major limitations: (1) they follow rigid, procedural workflows that underutilize the autonomous reasoning potential of LLMs, making it difficult to dynamically adapt testing strategies based on real-time feedback; and (2) they rely on rule-based context extraction that is not tailored to test generation, failing to capture fine-grained code dependencies and test-specific knowledge required for deriving test requirements. In this paper, we propose TestAgent, an LLM-based test generation approach that addresses the above limitations by emulating human testing practices via a multi-agent collaboration mechanism. Particularly, TestAgent designs three specialized agents, namely a requirement planner, a test generator, and a test reviewer, to simulate how developers understand, construct, and validate unit tests. To unleash the autonomous capabilities of LLMs, we equip TestAgent with a set of tool APIs that can be invoked dynamically in an on-demand and adaptive manner. To further support repository-level reasoning, TestAgent constructs a test-specialized knowledge graph via static analysis, which captures code entities and their dependencies across the project and persistently stores testing artifacts (e.g., test reports and failure analyses) produced during generation. Experimental results show that TestAgent achieves 97.46% execution rate, 92.34% line coverage, 90.24% branch coverage, and 83.69% mutation score on six Java projects, outperforming LLM-based baselines across all metrics and achieving substantially higher mutation scores than search-based tools.
Quanjun Zhang, Ye Shang, Siqi Gu et al.· 0 citations
Large Language Models (LLMs) have achieved remarkable progress in code generation, yet ensuring correctness in complex, repository-level tasks remains challenging. Existing approaches often use generated tests as static post-hoc validators, which limits their ability to guide implementation and may introduce misleading feedback when the tests themselves are incomplete or incorrect. In this paper, we introduce TDD-Agent, which operationalizes the test-driven development paradigm for code generation. TDD-Agent first prompts the model to generate executable tests, encouraging it to clarify expected behaviors before implementation, and then performs iterative dual-track refinement over both the generated code and tests using execution feedback. We first isolate the effect of test-first reasoning through a prompt variant TDD-prompt on LiveCodeBench, where it consistently improves upon reasoning-based prompting baselines. Building on this finding, we evaluate the full TDD-Agent framework on RepoEval, a repository-level benchmark, and show that it consistently outperforms retrieval-based and agent-based baselines. Additional analyses show that iterative refinement improves not only code correctness but also the effectiveness of the generated tests, yielding higher pass rates, coverage, and mutation scores, suggesting that tests can serve as evolving reasoning artifacts rather than fixed validators. Our source code is available at https://anonymous.4open.science/r/TDD-Agent-Framework-6370/.
Hong Yu, Ke-Shen Li, Jiakun Li et al.· 0 citations
While Transformer-based Large Language Models (LLMs) have advanced NLP, achieving an efficient automated development workflow in multi-agent systems remains challenging due to issues in code accuracy, testing effectiveness, and agent collaboration. To address these limitations, we propose UMLCoder, a novel multi-agent collaborative code generation framework designed to enhance software reliability and maintainability. UMLCoder comprises four specialized agents: a UML Expert Agent for generating precise structural diagrams to guide programming, a Code Generation Agent, a Test Case Generation Agent, and a Test Execution Agent for robustness verification. Benchmark evaluations demonstrate that UMLCoder achieves a pass@1 score of 71.9% on HumanEval and 70.3% on MBPP. Compared to baselines like GPT-3.5-turbo and LLaMA3, the proposed framework significantly improves code quality and computational efficiency, reducing the time complexity from $O\left(n^{2}\right)$ to $O(n)$ in specific scenarios.
Kehao Mao, Ruixi Lin, Guanyu Lu et al.· 2026 6th International Confe...· 0 citations
Large language model-powered coding agents are increasingly used to modify existing code repositories, for example, by adding features or fixing bugs. Yet existing repository-level benchmarks typically evaluate only whether the final patch passes tests. Satisfying a user request requires a long chain of interdependent reasoning and decisions: an agent must recover explicit and implicit requirements, formulate a repository-grounded implementation plan, and translate it into correct code. A pass/fail outcome cannot characterize how an unsuccessful trajectory diverges from the requirements and implementation process needed for a correct patch. To address this gap, we introduce SWE-RPG, a repository-level benchmark that combines executable patch evaluation with validated ground-truth references (GTs) for (1) Requirement Clarification and (2) Implementation Planning. These intermediate GTs support retrospective, GT-aligned diagnosis of complete coding-agent trajectories across clarification, planning, code generation, and artifact submission. SWE-RPG comprises 163 tasks from 31 Python and Java repositories, including 113 bug fixes and 50 feature additions. We evaluate 3 coding agents, including Claude Code, Codex, and OpenCode, with 6 large language model backends, including Claude-Sonnet-5 and GPT-5.6-Terra. Results show that the evaluated popular coding agents still struggle to implement user requests in existing repositories, achieving an average resolved rate of only 31.5% on SWE-RPG. Intermediate-GT diagnosis further identifies implicit requirement recovery as the main bottleneck, accounting for 24.5%--46.0% of agent runs. This result suggests implicit-requirement recovery as a key candidate direction for improving coding agents. The benchmark data and evaluation code are available at https://github.com/Xin-Zhou-smu/SWE-RPG-Bench.
Xin Zhou, Chun Yong Chong, Kisub Kim et al.· 0 citations
Large Language Models (LLMs) have revolutionized software development, from analyzing code and generating suggestions to detecting bugs and errors, and even creating entire programs. Despite these advances, existing AI-driven code review solutions still provide a one-size-fits-all approach to code review with overall feedback and suggestions, often of a non-specific nature. This restriction promotes modular architectures which would be able to provide specific and direct code quality reports. This paper presents the AgentCodeReview system, a multi-agent system that is able to conduct explainable code review and automated bug repair by leveraging software engineering agents with different code review tasks. There would be five independent entities, each one to be able to review code, analyze security, evaluate performance, document it and be able to automatically fix bugs. They run parallelly under the guidance of a centralized orchestration layer that collects the results from the analytical agents, calculates software quality scores and creates comprehensive HTML and PDF reports. Moreover, a Streamlit-based web interface was created that allows the interactive visualization of the results of the analysis and interactive entry of the input values. A set of twenty python programs was created to test the framework for effectiveness, consisting of a variety of runtime errors, security flaws, performance issues, documentation issues and a mixture of these types of errors. Two metrics, namely execution time and qualitative assessment were used to compare the proposed multi-agent framework with a single-agent framework as baseline. Experimental results demonstrated the benchmark execution success rate was 95%, while the multi-agent architecture provided more structured, explainable and domain specific feedback than the single agent. The extra computational cost of the coordinated analyses was acceptable for software quality assessment tasks because of the resulting interpretability and modularity. Through implementation and experiments, the results demonstrate AgentCodeReview's utility and extensibility to the field of explainable AI in software quality assurance. The proposed architecture can be expanded to other programming languages, integrated into the industrial development flow, and enhanced with the advanced LLMs for scalable intelligent code review.
Bharath Kumar N, T L Manasa· International journal of com...· 0 citations
Reproduction tests help developers confirm reported issues and provide executable feedback for issue resolution, yet issue reports in open-source projects rarely include such tests. Recent studies have explored generating issue reproduction tests from issue reports with large language models, but existing approaches largely rely on prompt-based pipelines that retrieve textual context and generate tests. This limits their ability to understand how reported issues behave in repository-scale codebases and to flexibly organize the construction of reproduction tests. In this paper, we propose ReProAgent, a multi-stage agent framework for reproduction test generation from issue reports. ReProAgent decomposes the task into four agent stages: bug localization, root cause analysis, test planning, and test generation. To support these stages, ReProAgent integrates task-specific tools for task decomposition and reflection, context retrieval from both textual sources and repository graphs, and runtime interaction with the execution environment. Experiments on SWT-bench-lite and SWT-bench-verified show that ReProAgent successfully reproduces 58.43% and 70.30% of issues, outperforming all baselines, with an average cost of $0.14 per instance. For example, when equipped with GPT-5-mini, ReProAgent exceeds OpenHands with the same backbone by 20.43 and 7.90 percentage points, respectively. ReProAgent also generalizes across multiple backbone LLMs and improves downstream issue resolution performance when integrated with existing repair approaches.
Quanjun Zhang, Yi Zheng, Ye Shang et al.· 1 citation