Jul 2026· ACM Transactions on Software Engineering and Methodology· 0 citations· 61 references
TL;DR
JSTestCraft reconstructs missing context via three enrichment agents: library, structural, and semantic via three enrichment agents, demonstrating that context reconstruction and agentic collaboration enable LLMs to perform more reliable and adaptive testing in dynamic JavaScript environments.
Abstract
Unit testing is crucial for software reliability in JavaScript, yet constructing comprehensive suites remains costly and error-prone. JavaScript’s dynamic features, flexible typing, asynchronous execution, and reliance on third-party libraries, pose persistent challenges for automated test generation. Although large language models (LLMs) show promise in code reasoning and synthesis, existing methods often miss the multilayer contextual information required for executable, semantically correct tests. This paper introduces JSTestCraft, an agentic, multi-layer framework for adaptive JavaScript unit test generation. JSTestCraft reconstructs missing context via three enrichment agents: library, structural, and semantic. These agents capture inter-function topology, third-party API semantics, and inferred type constraints in a shared contextual memory for reasoning-driven test synthesis. A testing and optimization layer generates, executes, and iteratively refines test cases. Evaluated on 20 real-world Node.js repositories, JSTestCraft outperforms state-of-the-art baselines, improving test pass rate by 60.9%, statement coverage by 14.2%, and branch coverage by 47.3%. Ablation analysis confirms each agent’s contribution to contextual completeness. Beyond metrics, JSTestCraft discovers 13 previously unknown bugs across 5 repositories, 6 of which received substantive acknowledgment from maintainers or community contributors. These results demonstrate that context reconstruction and agentic collaboration enable LLMs to perform more reliable and adaptive testing in dynamic JavaScript environments.
Test4Py is presented, a novel framework that enhances type correctness in automated test generation for Python by leveraging the program’s call graph to capture richer contextual information about parameters, and introducing a behavior-based type inference mechanism that accurately infers parameter types and constructs valid test inputs.
Runlin Liu, Zhe Zhang, Yunge Hu et al.· ACM Transactions on Software...· 0 citations
This paper addresses automated unit test generation with large language models (LLMs). LLM-based test generation has not yet attained a quality level sufficient for practical use in industry. Although LLMs often reproduce API syntax faithfully, they frequently disregard semantic usage constraints and execution-environment dependencies, leading to assertion failures, mock-related errors, and reference/resolution errors. A prior failure analysis of Java unit test generation using GPT-4o classified 2980 trials into eight failure patterns and identified three root-cause mechanisms: external context ignorance, internal context ignorance, and a syntax–semantics gap. Building on that analysis, this paper proposes a prompt design comprising three strategies: (1) making the execution state explicit in the generated test, (2) stating semantic constraints explicitly, and (3) injecting environment constraints prior to generation. In contrast to generic techniques such as few-shot learning or chain-of-thought prompting, each proposed strategy is tied to a specific root-cause mechanism, yielding a systematic design in which each rule is explicitly justified by its correspondence to a specific root-cause mechanism. Experiments on 298 methods with five models (GPT-4o, GPT-5, GPT-5.1-Codex, Claude Sonnet 4.5, and Gemini 2.5 Pro) show improved test execution success rates for every model, with absolute gains ranging from 1.1 to 21.1 percentage points (pp). Mock-related errors were reduced by 61.9%–99.2% relative to the baseline prompt, demonstrating effectiveness against the targeted failure patterns. Finally, conditions under which the strategies transfer to other code-generation tasks are discussed, along with limitations on their scope.
It is argued that semantic RAG is fundamentally unsuited for rule enforcement, and that Euclid-MCP can serve as a stable, shared reasoning substrate for both RAG-based assistants and agentic systems.
AssertMate is proposed, a novel agent-based assertion generation framework that enhances the quality and reliability of LLM-generated assertions through three key components: actual value construction that identifies assertion targets via static analysis and type-aware heuristics, and multi-perspective expected value prediction using code generation, retrieval-augmented generation (RAG), and chain-of-thought (CoT) reasoning agents.
Dong Wang, Qiaoyu Han, Lin Yang et al.· 0 citations
LLM-based software engineering agents are constrained by limited context windows: roughly 100K tokens must store structurally relevant code subsets to resolve bugs. Standard retrieval models treat code as plain text, forcing agents to resolve multi-hop dependencies including subclass chains, transitive callers and interface implementations through slow trial and error. We tackle this limitation with lossless knowledge compression, encoding source code into an OWL2 ontology to answer structural queries using minimal relevant code fragments. We present OwlPath, an OWL2 reasoning layer atop CodeGraph, a widely used code intelligence platform with 500K+ GitHub stars, offering a unified CLI for structural code retrieval. Powered by tree-sitter parsing, OwlPath supports multi-language repositories (Python, JavaScript, TypeScript, Go, etc.) and encodes language-specific semantics into a unified OWL2 ontology. It adopts two complementary modules. First, a transitive-closure engine fetches all structurally linked symbols via single SPARQL property-path queries, capturing multi-hop relations missed by string matching. Second, the OWL Software Knowledge Map (OWL-SKM) precomputes a compact 3KB summary with module trees, core APIs and issue-related symbols, directing agents to target modules in the first query. Evaluated on 18 SWE-bench Pro instances, OwlPath obtains a 68.4% strict-apply rate versus 66.7% for the CodeGraph baseline, cutting token usage by 28.8% and runtime by 39.5%. In offline retrieval tests over 67 instances, OwlPath improves recall 2.06 times (0.464 vs 0.226) and reaches 88.1% hit rate compared to CodeGraph's 59.7%. On a 37-question structural retrieval benchmark, recall rises from 4.4% to 28.8%, with 69-80% accuracy on transitive caller and interface tasks.
Bo Zhang, Renke Pan, Huan Chen et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
