An Empirical Study on the CI/CD Pipeline Evolution

Abstract

CI/CD pipelines have become an essential part of modern software development, enabling teams to deliver software faster, more reliably, and with less effort. While prior work has examined CI/CD adoption and workflow‐file changes, less is known about when higher level CI/CD activities are introduced during development, and how structural pipeline characteristics evolve across comparable development stages. This study analyzes most‐starred open‐source Java repositories on GitHub using GitHub Actions. We identify automated development activities and extract structural pipeline metrics from 339 repositories with CI/CD pipelines. We compare pipeline evolution across equivalent stages of development for 107 repositories. Using our activity‐detection procedure, build was the most frequently identified automated activity (93.5%), followed by release (44.8%), test (37.2%), analyze (22.7%), and deploy (14.2%). Build was typically detected earliest, with test and release following later, and analyze and deploy were detected in fewer projects and generally closer to the midpoint of active development. Pipeline‐modifying commits accounted for a mean of 4.15% of repository commits (median = 2.25%), suggesting that CI/CD pipelines require ongoing maintenance but that this maintenance represents a comparatively small share of overall repository commit activity in most repositories. Structural changes occur throughout development, with lines of code changing most frequently across timeline checkpoints, while secrets, comments, and conditionals show a secondary increase in later stages. Overall, the findings show that CI/CD pipelines evolve continuously and that both activity inclusion and structural pipeline changes follow recognizable lifecycle patterns.