SPINACH: Inferring Properties of Web Applications for Property-Based Testing
TL;DR
The paper asks whether high-level conceptual specifications improve LLM-generated PBT quality and whether they help developers extend and maintain AI-generated systems (RQ2), and reports preliminary results applying Spinach to two open-source applications.
Abstract
Turning AI-generated web application prototypes into reliable, deployable software requires testing, yet writing property-based tests (PBTs) is difficult: developers struggle to identify meaningful properties, and LLMs asked to generate tests from natural language documentation often miss relevant ones. We present Spinach, a VSCode extension that addresses this gap via a specification inference pipeline. An LLM agent first explores a web application and represents its functionality as software concepts, a user-facing behavioral specification. A second agent derives natural language properties from these concepts for developer review; a final phase translates approved specifications into LTL and executable PBTs using the Bombadil framework. We ask whether high-level conceptual specifications improve LLM-generated PBT quality (RQ1) and whether they help developers extend and maintain AI-generated systems (RQ2). We report preliminary results applying Spinach to two open-source applications, RealWorld and 4ga Boards, and outline planned evaluations using mutation testing and a user study of human-AI collaborative development.
