This work introduces Path2Spec, a divide-and-conquer framework that leverages LLMs to extract all execution paths from an input program, generates path-specific specifications for each, and merges them into a comprehensive overall specification.
Abstract
Formal specifications are critical for program verification, comprehension, and maintenance. However, manually writing them is costly and difficult to scale. Recent studies have shown that Large Language Models (LLMs) are promising for automated specification generation, but existing methods suffer from quality issues. We analyze a state-of-the-art approach and find that at least 34.6% of successfully verified specifications actually fail to meaningfully capture the program's distinct behavior, which is a quality issue not captured by metrics that only measure verification success. We further found that a major factor contributing to such hidden quality issues stems from the design of existing methods: these methods treat a program as a single unit, resulting in overly general, coarse-grained constraints. To this end, we introduce Path2Spec, a divide-and-conquer framework that addresses these limitations through systematic path-based reasoning. Path2Spec leverages LLMs to extract all execution paths from an input program, generates path-specific specifications for each, and merges them into a comprehensive overall specification. For complex programs where path-based generation struggles, Path2Spec employs a decompose-then-retry strategy that recursively breaks a program into smaller subprograms based on logical branches, generates specifications for each, and merges them back. We evaluate Path2Spec on two public benchmarks: SG-Bench (120 programs) and SV-COMP (265 programs). Results show that Path2Spec can outperform the state-of-the-art baseline SpecGen: 87.5% versus 66.7% on SG-Bench, and 83.0% versus 44.2% on SV-COMP. Human evaluation further validates that Path2Spec generates higher-quality specifications with precise semantic alignment to the code.
Automated unit test generation promises to reduce the cost of software quality assurance, and hence, is attracting attention from both academia and industry. Yet, generating assertions that are executable, meaningful to developers, and able to catch faults remains an unsolved challenge. Existing approaches either rando...
Jia-Lun Cao, Hao-Yu Wang, Hao-Ran Yan et al.· Proceedings of the ACM on So...· 0 citations
Automatically generating high-coverage unit tests for complex Java methods remains a formidable challenge, particularly when execution paths are guarded by intricate control-flow nesting and cross-class state dependencies. Existing LLM-based approaches predominantly follow a goal-driven paradigm, relying on unguided co...
Rui-Guo Yu, Rui-Qi Dong, Xi Xiao et al.· Proceedings of the ACM on So...· 0 citations
This dissertation aims to make specification recovery from imperfect code accurate, evidence-driven, and uncertainty-aware, so that it consistently yields effective bug-finding tests.
Jun-Da Zhao· Companion Proceedings of the...· 0 citations
Recent advances in large language models (LLMs) for code generation have given rise to vibe coding, a programming practice in which developers interact with LLMs through unstructured natural language and primarily validate implementations by inspecting overall execution outputs. However, the inherent ambiguity and inco...
Rui-Rui He, Han-Qing Lu, Le-Zhi Ma et al.· Proceedings of the 41st IEEE...· 0 citations
This paper proposes CAST, a systematic testing framework designed to evaluate the compositional safety of LLMs in the domain of malicious code, and demonstrates that CAST systematically exposes severe safety violations in strongly aligned models that resist conventional red-teaming.
Lu Yan, Zhuo Zhang, Xiang-Zhe Xu et al.· Proceedings of the ACM on So...· 0 citations
This work presents SpecRead, a benchmark that isolates specification comprehension from generation ability, and is automatically scorable by deterministic checks, with gray-zone cases counted wrong under the conservative main scoring.
Fei-Lian Huang· arXiv.org· 0 citations
Related blog posts
Microsoft Research Blog· microsoft.comNov 16, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT scientists developed a training method to generate models that simulate the effects of human activities on climate much more quickly than other approaches.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.