Trust the Delta: Rethinking Software Validation in the AI Era (Keynote)
TL;DR
This keynote argues that software changes offer a uniquely tractable setting for establishing correctness, and reframes the oracle problem from determining absolute correctness to validating a behavioral delta by asking which behaviors changed, which remained stable, and whether the observed differences match the intended change.
Abstract
Generative AI is making software changes increasingly cheap to produce, shifting the bottleneck from implementation to validation. This keynote argues that software changes offer a uniquely tractable setting for establishing correctness. Unlike an arbitrary program, a change comes with an existing implementation whose behavior is largely accepted and, often, a natural-language description of what is intended to change. Together, these provide a powerful partial specification in which existing behavior should be preserved by default, except where the stated intent justifies a difference. This reframes the oracle problem from determining absolute correctness to validating a behavioral delta by asking which behaviors changed, which remained stable, and whether the observed differences match the intended change. By combining natural-language intent with execution across program versions, AI-based techniques can construct increasingly strong evidence about the correctness of a change. As coding agents become more capable, the central challenge for AI4SE therefore shifts from generating patches to justifying them. Future systems should produce not only code changes, but executable evidence that the intended change, and only the intended change, actually occurred.