Jul 2026· Annual International Computer Software and Applications Conference· pp. 2126-2131· 0 citations· 20 references
Abstract
Autonomous AI agents are increasingly being used to assist software developers in tasks such as code generation and refactoring. However, we still lack a clear understanding of how these agents interact with humans in collaborative workflows such as GitHub pull requests (PRs). A helping hand to study this phenomenon comes from the AIDev dataset featuring 932k PRs authored (i.e., opened) by AI agents across 116k GitHub repositories. Nevertheless, such a dataset provides a limited view of the collaborations between humans and agents, since the latter may also be involved in PRs opened by humans. In this work, we analyse interaction patterns between humans and AI agents in PRs. To support this analysis, we introduce a novel methodology that extends the AIDev dataset to capture all PRs involving agent participation, irrespective of whether the agents initiated the PR. We apply such a methodology to a subset of 383 repositories from AIDev, finding 6,445 PRs, out of which 5,118 are agentrelevant (vs the 3,197 of AIDev on the same repositories) for a total of 74,525 interactions from 671 unique users. We classify these PRs by authorship (i.e., agent, bot, or human) and examine which participant types perform key interactions (e.g., reviews). Our findings indicate that human-agent collaboration is the most common interaction pattern, with about nine out of ten changes being merged. Although agents author a substantial proportion of PRs, fully autonomous contributions remain uncommon and are rarely accepted. These results highlight the increasing importance of AI agents in augmenting human capabilities in software development, as well as the central role of human oversight in ensuring successful contributions.
The AIDev-pop dataset is used to provide the first empirical examination of the prevalence of concurrent submission using PRs authored by agents, and a classification system based on the detection of conflict reported by git is developed.
George Xu, Arjun Subramanian, N. Karthik· arXiv.org· 0 citations
Large-scale human–agent collaboration is investigated by comparing 40,214 pull requests across 2,807 GitHub repositories, including 33,596 agent-authored PRs from five autonomous coding agents and 6,618 human-authored PRs, revealing a socio-technical trade-off.
A. Njoku, Zohreh Sharafi, F. Khomh· AIware· 2 citations
Papers that address the orchestration of teams by synthesizing their workflows into a coherent whole, whether these teams are composed of human, machine, Generative AI (gen-AI), robot or AI-Agentic members are proposed.
W. Lawless, Marco Brambilla, Stephen Russell· AHFE International· 0 citations
Analysis of agent-generated PRs from the AIDev dataset studies how often agents include test changes and how well covered are code changes by existing and agent-written tests, finding error-handling constructs are the most consistently under-tested.
A. Dipongkor, Talank Baral, Wing Lam et al.· arXiv.org· 0 citations
AI coding agents are increasingly integrated into software development workflows, operating on both sides of the pull-request (PR) process: AI authoring agents create or modify PRs, while AI reviewers evaluate them. This creates a closed loop in which one AI coding agent reviews a contribution attributed to another. We construct a large-scale dataset of AI-to-AI code review by linking AI-attributed PRs with AI-attributed review events from CodAGE, a public dataset of coding-agent-generated GitHub events. Our dataset contains 248,641 unique AI-attributed PRs that received at least one AI-attributed review. Of these, 45,269 received cross-product review and 208,145 received same-product review; 4,773 PRs received both. Cross-product AI-to-AI review occurred in approximately 1.6% of identified agent-authored PRs but was substantial in absolute terms, and its volume increased by more than two orders of magnitude from 2025-Q1 to 2025-Q3. Reviewer output varied across author-reviewer configurations. CodeRabbit labeled 35.0% of its comments on Claude Code-authored PRs as refactor comments, compared with 10.5% on Copilot-authored PRs, although this difference may reflect characteristics of the PRs rather than the reviewer. For three of four dual-role reviewers, mean comments per PR were 58-65% higher in the same-product group, although effect sizes were small or negligible and the difference was concentrated in the upper tail. Among pairs with complete, nonnegative timestamps, the observed median latency was 1.2 minutes for cross-product pairs and 4.7 minutes for same-product pairs; differential timestamp availability and reviewer composition limit this comparison. Overall, closed-loop AI-to-AI review is increasing but remains a minority of identified agent activity, with review output varying across authoring-agent groups and product configurations.
AI agents are becoming a fundamental part of modern software creation, helping developers in generating code, debugging, designing systems, etc. But there is a clear difference between how beginners and experienced software engineers get benefits from these tools. Newbies usually depend on agents for one-time prompts and quick answers, whereas mature users utilize them through well-defined, repeated workflows that raise productivity and consistency. In this article, we discuss this difference and emphasize that getting the full potential does not merely depend on better prompts but on workflows driven by instructions developers create clear and reusable instruction files to direct agent behavior across tasks. When developers stop seeing agents only as chat interfaces but as programmable collaborators, they can produce more reliable and high-quality outputs. We offer in our paper methods like designing modular instructions, narrowing down the context, and iterative refinement loops, as well as a case study illustrating how a team made a code review more efficient and minimized the rework by making agent instructions standard. The results stress that structured forms of interaction rather than sporadic use are the main ways to tap into advanced features. Our paper provides a conceptual model for agent usage at large scale, hands-on advice for the implementation of instruction files in actual settings, and validation that skillful developers can far exceed basic usage by adopting orderly, system-like approaches to agent collaboration.
Madhurima Kommuru, Srujana Pulipaka· International Journal of Mod...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
