Skip to content

Specification Portability Across LLM Development Agents: Cross-Agent Compatibility in Specification-Driven Software Migration

Aug 2026 · 1 citation · 27 references
Computer Science

TL;DR

The results show that specification size alone does not predict implementation quality and that cross-agent transfer can produce substantial agent-dependent degradation, and suggest that specifications in heterogeneous SDD workflows should not automatically be treated as agent-neutral artifacts.

Abstract

This paper investigates cross-agent specification portability using Oracle-to-PostgreSQL migration as a controlled software transformation task. The study combines two experimental stages. First, a specification-first migration pipeline was evaluated on 1,006 PL/SQL files, of which 623 were successfully regenerated and 380 generated scripts executed successfully in PostgreSQL 16. Second, cross-agent experiments were conducted on a dataset of 1,802 Oracle scripts with corresponding PostgreSQL implementations using Amazon Kiro, Google Gemini, and GitHub Copilot, with Claude Code and Cursor included in the initial single-agent evaluation. Native and foreign specifications were assessed using Token F1, exact match, SQL syntax validity, AST exact match, AST mean similarity, and immediate runnability. The results show that specification size alone does not predict implementation quality and that cross-agent transfer can produce substantial agent-dependent degradation. The strongest replicated case occurred when Gemini directly consumed a Kiro-origin specification, producing a Token F1 of 0.035, SQL syntax validity of 2.33%, and AST mean similarity of 0.015. Rewriting substantially improved Gemini in the tested configuration, compression did not provide a universal benefit, and retrieval-augmented ingestion was the only common strategy represented on the per-agent Pareto frontiers of both Gemini and Copilot. The findings suggest that specifications in heterogeneous SDD workflows should not automatically be treated as agent-neutral artifacts and motivate explicit consideration of specification portability, agent-specific interpretation, and retrieval-based access in multi-agent software engineering.

View source

Similar papers

Jul 2026

Specification-Driven DevOps for Multi-Service Environments

This study investigates whether a frontier LLM can generate Dockerfiles and Docker Compose configurations for multi-service applications using repository contents without access to developer-authored deployment artifacts and analytically derives a minimal explicit deployment specification for information that cannot be...

Oleg Grynets, Kyrylo Fursov, V. Lyashkevych et al. · 1 citation
Conference Jul 2026

Metamorphic Testing of Multi-Agent LLM Systems: A Trace-Based Behavioral Oracle Framework

Multi-agent systems built on large language models (LLMs) are increasingly deployed for complex tasks requiring autonomous planning, tool use, and inter-agent coordination. However, the non-deterministic nature of LLM outputs and the emergent behavior arising from agent interactions render traditional test oracles inef...

Gopalakrishnan Marimuthu · 0 citations
Conference Open access 2026

Chasing Abpms Vision: The Bab Framework Approach for LLM Agents Over Model-driven Enterprise Systems

A runtime architecture that extends the BAB framework on Jakarta EE to support agent-augmented applications hosted alongside ontological-driven generated cores and an architectural pattern that lets model-driven rigor and LLM agent flexibility coexist in enterprise environments without compromising either is introduced...

Borivoj Bogdanović, M. Segedinac, Z. Konjović et al. · 0 citations
Preprint Jul 2026

From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents

A reusable engineering pattern for turning exploratory prototypes into auditable applications with versioned source, control, and validation artifacts is presented, which reconstructs this pattern into a traceable, auditable LLM-agent architecture.

Joongho Ahn, Moonsoo Kim · 0 citations
Jul 2026

TraceDev: A Traceability-Driven Multi-agent Framework for Requirement-to-Code Development

This work proposes TraceDev, a multi-agent framework for automated software development grounded in use cases that contain multiple functional points and complex semantics, and demonstrates the effectiveness of TraceDev in repository-level code generation from requirements.

Mingyu Chen, Ya-Kun Zhang, Zihao Xie et al. · 0 citations

Related blog posts

Microsoft Research Blog Oct 12, 2026

Simulating student learning for better AI tutoring

StudentSim is a research system for evaluating and improving AI tutoring tools before they are tested with learners. It models how individuals may engage with tutors, giving feedback to improve tutor performance across subjects like chess, writing, and mathematics. The post Simulating student learning for better AI tutoring appeared first on Microsoft Research.

MIT News · Artificial Intelligence Oct 2, 2026

Documenting the tech worker movement

Writing as a participant and researcher, PhD student JS Tan SM ’22 has co-authored a new book about the rise of tech worker protests and the employer backlash that followed.

MIT News · Artificial Intelligence Oct 2, 2026

Documenting the tech worker movement

Writing as a participant and researcher, PhD student JS Tan SM ’22 has co-authored a new book about the rise of tech worker protests and the employer backlash that followed.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.