Skip to content

AI-powered Code Review with LLMs: Early Results

Apr 2024 · arXiv.org · Vol abs/2404.18496 · 62 citations · ⚡ 3 influential · 26 references
Computer Science

TL;DR

The goal is to not only refine the accuracy of the LLM-based tool but also to underscore its potential in streamlining the software development lifecycle through proactive code improvement and education.

Abstract

In this paper, we present a novel approach to improving software quality and efficiency through a Large Language Model (LLM)-based model designed to review code and identify potential issues. Our proposed LLM-based AI agent model is trained on large code repositories. This training includes code reviews, bug reports, and documentation of best practices. It aims to detect code smells, identify potential bugs, provide suggestions for improvement, and optimize the code. Unlike traditional static code analysis tools, our LLM-based AI agent has the ability to predict future potential risks in the code. This supports a dual goal of improving code quality and enhancing developer education by encouraging a deeper understanding of best practices and efficient coding techniques. Furthermore, we explore the model's effectiveness in suggesting improvements that significantly reduce post-release bugs and enhance code review processes, as evidenced by an analysis of developer sentiment toward LLM feedback. For future work, we aim to assess the accuracy and efficiency of LLM-generated documentation updates in comparison to manual methods. This will involve an empirical study focusing on manually conducted code reviews to identify code smells and bugs, alongside an evaluation of best practice documentation, augmented by insights from developer discussions and code reviews. Our goal is to not only refine the accuracy of our LLM-based tool but also to underscore its potential in streamlining the software development lifecycle through proactive code improvement and education.

View source

Similar papers

Review Aug 2026

Generative AI use in Statistical Research: A Literature Review and Code Generation Case Study

GenAI can function as a research tool, but not as a substitute for methodological expertise, and has potential to increase efficiency of tasks which take advantage of its search and summarization abilities, as well as basic code debugging and algorithm formation.

Natalie Morosin, A. A. Nadi, Michael P Wallace · 0 citations
Preprint Aug 2026

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code

It is observed that generated code often omits basic input validation or memory-safety checks, which can lead to overflows, resource exhaustion, or other reliability/security issues, and even the largest models frequently make simple mistakes.

Rodrigo Pato Nogueira, Marco Vieira, João R. Campos · 0 citations
Review Jul 2026

From Code Review to Code Critique: Intent, Drift, and Spotlight for AI-Generated Diffs at Scale

AI coding agents are generating code at volumes that exceed the capacity of traditional peer review. At the same time, existing AI code review tools over-index on low-value suggestions such as style and best practices while under-indexing on the concerns human reviewers prioritize most: correctness, security, and performance. We present ARCTIC, an AI-powered Code Critique system that reframes code review around three capabilities: intent prediction, which infers why a change was made from conversation logs and metadata; drift detection, which measures divergence between the developer's intent and the agent's output via backtranslation; and code spotlight, which ranks the regions of a diff most warranting human scrutiny. We ground these capabilities in a six-theme taxonomy derived from 18,000 code reviews. Offline evaluation shows that intent prediction achieves 0.86 F1, drift detection reaches near-perfect ordinal agreement with human annotators (QWK = 0.907), and spotlight outperforms the baseline AI reviewer by 2.4x on quality estimation at 5x fewer tokens. In the experimental rollout, the drift scores reduces code misalignment by an additional 5.76 points (p = 0.026), intent prediction receives 90.2% approval, and zero defects have been attributed to self-reviewed diffs since launch.

C. Maddila, Mashrur Rashik, E. Khan et al. · 0 citations
Conference Jul 2026

Understanding and Reducing Code Smells in AI-Generated Java Code Using Explainable AI and Prompt Optimization

Recently, Developers have been relying on AI tools to support them in their daily work by generating code. While the use of large language model-based AI tools has improved productivity, the quality of the generated code wasn't always optimal. In a lot of cases, the code includes design issues known as code smells, which negatively impact readability, maintainability, and future development. This paper investigates these issues in AI-generated Java code, with a focus on common object-oriented problems such as switch statements, temporary fields, and refused bequest. A structured approach is proposed that combines static analysis tools with explainable AI techniques to better understand why these problems appear. Based on the realized insights, prompts are optimized to guide the AI model towards generating cleaner and more structured code. The results showed clear improvement after the prompt optimization, where the number of detected code smells was reduced by 66%, and completely removed (100% reduction) in some cases. Overall, the study showed that improving prompt design, supported by explainable analysis, can significantly enhance the quality of AI-generated code.

Y. Younes, Yousef Elsheikh · 0 citations
Review Open access Jul 2026

A Systematic Literature Review on Automated Program Repair using Large Language Models

Current research is summarized to identify key gaps and future directions to optimize LLM based APR are proposed, to assure its reliability and scalability in real world software development.

Fatmaelzahra Hamdi, Ramadam Moawad, A. Mohsen · 0 citations

Related blog posts

MIT News · Artificial Intelligence Jun 3, 2026

MIT researchers teach AI models to interpret charts

The new ChartNet training dataset could improve the accuracy of vision-language models that help analyze business trends or interpret scientific figures.

Microsoft Research Blog Jul 30, 2026

Echoverse: Deep, evolving environments for computer-use agents

Computer-use AI agents struggle with multi-step workflows like email and customer support. Echoverse trains agents in realistic environments rather than simply providing more training tasks, helping them improve as the tasks, tests, and environments evolve. The post Echoverse: Deep, evolving environments for computer-use agents appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.