Regression Bugs in Post-Release Github Issue Reports: A Pilot Study

Abstract

Software systems continuously evolve through feature additions, bug fixes, refactoring, and other developer-driven changes. Although these changes are intended to improve the quality of the software, they can also unintentionally disrupt behavior that previously worked as expected. Such failures are known as regression bugs: defects in which an existing feature, workflow, or system behavior becomes broken, degraded, or incorrect after a software change. GitHub issue trackers represent a rich stream of crowd-sourced stakeholder feedback that captures failures in implicit behavioral continuity, making them a candidate source of post-release requirements validation evidence. Despite the availability of automated regression bug detection tools that rely on test cases to prove that a behavior has changed, many regressions observable through crowd feedback are not encoded in developer tests and therefore remain outside the reach of purely algorithmic detection. In this study, we apply a structured multi-artifact classification guideline — synthesizing evidence from issue reports, developer discussions, pull requests, commit histories, and release notes — to identify and classify regression bugs in 100 issue samples from the Mastodon v4.3 release. We also identify recurring patterns in crowd-visible regressions that are not captured in developer tests. These findings advance the understanding of regression bugs in open-source software (OSS) and offer practical implications for developers, requirements engineers, and tool builders seeking to improve regression detection through crowd feedback.