Log in
Back to #software testing
#software testing Preprint

Investigating Software Aging in LLM-Generated Software Systems across Generation-and-Execution Environments

Cesar Santos Michele Vitagliano Roberto Natella Ermeson Andrade
Aug 2026 · 0 citations · 38 references
Computer Science

TL;DR

It is found that functional correctness alone is insufficient to assess the operational reliability of LLM-generated software before deployment in continuously running environments, and aging trends can also emerge in manually developed implementations.

Abstract

Large Language Models (LLMs) are increasingly used to generate executable software systems from natural language specifications, accelerating development and reducing manual implementation effort. Although recent studies have investigated the functional correctness, security, maintainability, and robustness of LLM-generated code, little is known about the long-term reliability of such systems under sustained execution. In this paper, we experimentally investigate software aging symptoms in LLM-generated service-based applications across different programming languages. Using backend scenarios derived from BaxBench, we generated applications targeting JavaScript, Python, and Rust through LLM-based generation platforms, validated them with BaxBench-derived tests, and subjected them to 48-hour workload executions. We monitored memory usage, response time, and throughput and analyzed them using the Mann--Kendall test and Sen's slope estimator. We further complemented the runtime evaluation with static analysis of the generated source code and an exploratory comparison with human-written implementations of related backend scenarios. The results show that memory usage is the most consistent indicator of potential software aging, with statistically significant upward trends in most application-language combinations, while response time and throughput exhibit more heterogeneous behavior. Static analysis identified plausible code-level aging mechanisms, and the comparison with human-written systems showed that aging trends can also emerge in manually developed implementations. These findings indicate that functional correctness alone is insufficient to assess the operational reliability of LLM-generated software before deployment in continuously running environments.

arXiv Semantic Scholar
View source

Similar papers

#software testing Preprint Aug 2026

Evaluating Inference-Time Defenses Against Package Hallucination in LLM-Generated Code

LLMs are increasingly used for code generation, yet they frequently hallucinate non-existent software packages, creating exploitable entry points into the software supply chain. We make four contributions to this problem. First, we show that prior evaluation methodologies systematically inflate hallucination rates by misclassifying standard-library modules as hallucinations in some languages. For Python, the overestimation reaches 9.4 percentage points. Second, we evaluate seven inference-time defenses for mitigating package hallucinations, including five guided decoding strategies (Greedy, Contrastive, DoLa, Nudging, and Active Layer-Contrastive Decoding), an iterative self-refinement approach (Self-Refine), and a Retrieval-Augmented Generation (RAG)-based defense.. Across eight models spanning five families and four programming languages (Python, JavaScript, Ruby, Rust), RAG reduces the package hallucination rate (PHR) in 18 of 32 model--language configurations. Third, we introduce Package Utility (PU) to assess whether defenses preserve valid and task-relevant recommendations. Among strategies evaluated, Greedy decoding provides the strongest average mitigation--utility trade-off. Fourth, we stress-test all strategies under adversarial prompts seeded with fabricated package names and find that PHR surges by up to 45 percentage points relative to standard prompts, with Ruby consistently the most vulnerable language (80.9--95.2\%). Under adversarial conditions, RAG and Self-Refine outperform all decoding-only strategies, indicating that robust defense requires either external grounding or iterative self-verification when prompts are actively hostile. Our results recast package hallucination as both a measurement problem and a decoding-time control problem, and they demonstrate that the choice of defense must be matched to the threat model and recommendation utility.

Albérick Euraste Djiré, Iyiola E. Olatunji, Melissa Tessa et al. · 1 citation
#software testing Review Aug 2026

Model-Based Agentic Software Engineering

MAGE explains how externalized knowledge, bounded action, independent evaluation, and retained human authority can compose into a governed engineering environment, and proposes tests of when that environment turns commodity intelligence into durable engineering progress.

James C. Davis, Kelechi G. Kalu, Huiyun Peng et al. · 1 citation
#software testing Review Aug 2026

"A Second Set of Eyes": The Process and Challenges of Software Documentation Review

The work identifies five distinct stages of the documentation review process: self review, technical review, editorial review, play testing, and post-publication feedback, and draws on practitioners with distinct expertise to address quality across content, presentation, and user experience.

Avinash Bhat, Ian Arawjo, Disha Shrivastava et al. · 0 citations
#software testing Sep 2026

Mixed Reality Glasses Image Translocation for Binocular Diplopia.

This prototype MRG image translocation software was helpful to 69% of patients with binocular diplopia, but limited by large angle strabismus because of the limited instrument field of view.

Edsel B Ing, Kevin Sha, Sarosh Dandoti et al. · 0 citations
#software testing Open access Aug 2026

Multi-Disease Prediction Using Machine Learning: A Web-Based Diagnostic Support System for Diabetes, Heart Disease, and Parkinson\'s Disease

A diagnostic support system based on a unified web platform that classifies patients according to the risks of developing three diseases based on regularly collected clinical or audio data using classical supervised learning algorithms is presented.

Vedamurthy D R, Dr. Anup Ritti, A. Bibi et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Aug 17, 2026

Q&A: Rethinking how innovation happens

In his latest book, Professor Eugene Fitzgerald examines the forces that turn breakthroughs into value — and why innovation resists simple formulas.