Neural Program Modeling Learning Program-Specific Models that Predict Execution Features from Input Features and vice versa

Abstract

Understanding the semantics of program code is a fundamental problem in software engineering, posing continuous challenges for testing and analysis, as well as for AI agents. We introduce neural program modeling: an approach that learns the relationship between input features and execution features (such as traces or coverage) from pairs of generated system inputs and monitored execution data. The resulting program-specific model can predict system inputs that trigger desired program behaviors—in seconds, without executing the program. It can also predict program traces and coverage for a given input, again without running the program. Once trained, such a model acts as a proxy for the program under test, supporting a range of tasks: (1) Predicting inputs for specific execution features, guiding test generators and developers; (2) Replacing the original program during search-based testing or fuzzing, reducing costly or risky executions; (3) Predicting which program parts a test input will execute, guiding test selection; (4) Predicting which input will trigger a specific behavior, serving as an input filter to prevent undesired outcomes. To our knowledge, this is the first work that trains reversible, program-specific neural machine translation models to both predict execution behavior from inputs and synthesize inputs to achieve the requested behavior in real-world programs. In our evaluation of real-world programs, including Bottle and the Google re2 library, we show that neural program modeling can predict execution features for unseen inputs, achieving similarity ratios ranging from 91.57% to 98.59%. We can also predict program inputs that trigger specified behaviors with accuracy and recall up to 100% after validation and refinement. All predictions can be automatically validated and refined, either by retraining the model on new input-trace pairs or by using a backpropagation loop to adjust its parameters. Model training and inference are resource-efficient and can be performed on commodity hardware.