Can LLMs Detect and Refactor Test Smells in Performance Tests?

Abstract

Performance testing is essential to ensure that software meets its responsiveness and scalability requirements, and tools such as k6 are used to write performance test scripts that exercise these aspects in a structured and optimized way. Like any code, regardless of the programming language, k6 scripts are subject to recurring bad practices, known as performance test smells, that can yield misleading results or waste resources. While Large Language Models (LLMs) have shown promise in detecting and refactoring conventional test smells, their effectiveness on performance test smells remains largely unexplored. This paper evaluates whether three widely used LLMs (Claude, GPT, and Gemini) can detect and refactor four performance test smells in real-world k6 scripts, and whether adding illustrative code examples to the prompt improves their performance. Using a dataset of 228 k6 scripts collected from 58 GitHub repositories and labeled by a static linter, we submitted 1,368 detection-and-refactoring queries across two prompt variants and manually reviewed 120 of the resulting refactorings. We find that detection performance is moderate and comparable across the three models (F1 micro between 0.49 and 0.54, and between 0.53 and 0.54 under the definition-only prompt), that adding code examples does not consistently improve detection and can even degrade it, and most notably that detection difficulty and refactoring difficulty are largely decoupled: the smells that are hardest to detect are often refactored almost flawlessly once found, whereas one of the most reliably detected smells accounts for nearly all incorrect and breaking refactorings. We attribute this asymmetry to a distinction between declarative and architectural fixes, and discuss its implications for trusting LLM-generated refactorings in performance testing.