Skip to content
Log in
Sponsored AI Networking Cookbook: Practical recipes for AI-assisted network automation and development AI Networking Cookbook: Practical recipes for AI-assisted network automation and development

SemaDiff: Identifying Semantic-Changing Commits with Generated Code and Tests

Maha Ayub Michael Konstantinou Ahmed Khanfir Nikolaos Tsantalis Mike Papadakis
Jul 2026 · arXiv.org · Vol abs/2607.13111 · 0 citations · 42 references
Computer Science

TL;DR

SemaDiff, a novel approach for identifying semantic-preserving commits through behaviour-based analysis, is proposed, and results show that SemaDiff distinguishes accurately semantic-preserving from -- changing commits in about 76% of the cases, with a 100% precision in semantic-changing commit detection.

Abstract

Distinguishing semantic-preserving commits from changing ones remains an open challenge in software repository mining. While existing approaches detect refactoring commits accurately, they cannot ensure that a commit is purely semantic-preserving, without any interleaving behaviour-changing modification. This limitation can impact several tasks, such as debugging, fault localisation, bug dataset construction, rollback analysis, and bug fixes backporting. To fill this gap, we propose SemaDiff, a novel approach for identifying semantic-preserving commits through behaviour-based analysis; comparison of similar test execution on pre- and post-commit versions. As code impacted by the refactoring is often hard to test and different accross both versions, we propose generating additional calling methods to that code, which serve as testing target. Given a commit, SemaDiff analyses the diff to identify modified code and extracts unchanged dependent code that calls it. It then generates an additional dependent class using a large language model to exercise the changed code in both versions, and automatically generates tests for the dependent code. This way, we obtain the same tests for the different code versions, enabling the behavioural-difference detection. The commit is classified as semantic-preserving only if all generated tests produce identical outcomes across the two versions. To evaluate SemaDiff, we construct and annotate manually a dataset of 183 commits, gathered from well-known open-source Java projects. The obtained results show that SemaDiff distinguishes accurately semantic-preserving from -- changing commits in about 76% of the cases, with a 100% precision in semantic-changing commit detection.

DOI arXiv Semantic Scholar
View source

Similar papers

Review Jul 2026

From Discussion to Execution: Replicating Buggy and Correct Data Science Code

Reprodgen is introduced, a large language model (LLM) based framework for automatically replicating executable buggy and patched data science programs from Q&A forum posts, and results show reliable replication with clear differences in model performance.

Ragib Shahariar Ayon, Mohammad Wardat, Shibbir Ahmed · 0 citations
Book Open access Jul 2026

When Testing Meets Refactoring: Catalogue, Detection, and Recommendation

This paper presents a unified catalogue of test refactoring patterns derived from a mixed-method approach involving repository mining, developer surveys, and analysis of StackOverflow discussions, to establish a rigorous benchmark for evaluating future automated recommendation systems, including those driven by Large Language Models.

Victor Guerra Veloso · 0 citations
Conference Jul 2026

Commit-Message-Augmented Static Analysis of Cross-Site Scripting Fixes in Java Web Applications

Cross-Site Scripting (XSS) remains a common and high-impact web vulnerability. Detecting XSS with high precision is difficult because exploitability depends on end-to-end data flow (sources, sanitizers, and sinks), framework-specific rendering semantics, and the correctness of context-aware output encoding. Although secure development lifecycles increasingly integrate static application security testing (SAST) into CI/CD pipelines, practical adoption is often limited by false positives and incomplete modeling of project-specific sanitization and templating behavior. In parallel, machine learning (ML) approaches show promise for learning vulnerability-relevant patterns from code, but they depend on the availability of realistic labeled data. This paper studies a low-cost repository-native signal that is routinely available but underused in ML-assisted static analysis: commit messages. We propose a mining and weak-labeling pipeline for Java repositories that identifies candidate XSS-fix commits using message heuristics grounded in OWASP guidance and CWE taxonomy, aligns commit messages with pre-/post-change code contexts extracted from diffs, and trains classifiers under two feature settings: (i) code-only and (ii) code+commit-message fusion. Across three model families (Random Forest, TextCNN, and BiLSTM), commit-message fusion consistently improves F1 and AUPRC and reduces false positives, suggesting that developer intent provides a useful auxiliary signal for security triage.

Abu Alam, Qka Mirza · 0 citations
Review Aug 2026

COMMITGUARD: Differential Slice Fuzzing for Commit-Induced Bug Detection

The key insight behind COMMITGUARD is that the pre-commit version of a modified function can serve as a behavioral baseline for interpreting bugs found after the commit, and a commit-aware differential slice-based fuzzing approach for verifying code changes is introduced.

Aniruddhan Murali, N. Mathews, Mahmoud Alfadel et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.