Generative AI is increasingly permeating software engineering, enabling developers to generate functions, files, and even entire applications from natural language specifications. AI systems are also becoming more personalized, adapting outputs based on inferred user characteristics and interaction history. While personalization may improve the development experience, it raises concerns that generated software could be shaped by attributes of the developer rather than by task requirements alone. Prior work has shown that generative AI can produce biased software artifacts, but little is known about how developer identity can bias generated code. We characterize three dimensions through which inferred developer attributes can influence generated artifacts: interface design, template content, and code structure. First, through controlled experiments on 800 AI-generated websites, we find that age- and gender-related signals produce significant differences across all three dimensions. Second, we conduct an observational study and follow-up interviews with 20 participants who used AI to create a personal website to both examine how personalization impacts software artifacts in practice, and also to understand how programmers perceive the boundary between personalization and bias. Together, our results show that developer attributes can meaningfully influence generated software beyond stated requirements, highlighting a previously underexplored tension between personalization and fairness in AI-assisted programming.
ABSTRACT Generative AI (GenAI) has advanced rapidly and made significant impacts. However, its effect on developers remains a topic of industry debate. Companies want to know whether GenAI can enhance developers’ coding performance, as an unclear understanding may put companies at a disadvantage. While the literature has begun addressing this issue, a formal understanding of GenAI’s impact remains incomplete. Moreover, existing findings are often short-term, fragmented, or lack explanatory mechanisms. To fill these gaps, we designed a multimethod research program comprising a longitudinal field study and a randomized controlled experiment. In Study 1, we collaborated with a global information technology organization and applied a difference-in-differences approach to over 27 weeks of proprietary data. In Study 2, we designed a randomized experiment involving 253 software developers. From these studies, we find that GenAI usage affects both developers’ coding quantity and quality. These effects, however, depend critically on how the tool is used. While reduced cognitive effort can be associated with diminished quality, interestingly, GenAI usage enables developers to produce higher-quality code with less cognitive effort. In this current study, we explain the paradoxical findings through cognitive load theory, showing that GenAI reduces extraneous load while preserving germane processing during ideation and debugging. Using a multimethod research design that integrates longitudinal field data with a randomized controlled experiment, we link observed performance effects to underlying cognitive mechanisms and usage strategies. We also offer guidance on effective usage styles and propose boundary conditions for realizing GenAI’s benefits in practice.
Guohou Shan, Michael Rivera, Subodha Kumar et al.· Journal of Management Inform...· 0 citations
The first large-scale, cross-platform study of plugins from five major web application marketplaces, covering domains from office productivity to software development, indicates that AI-assisted plugins face a range of emerging issues that negatively impact user experience and fail to comply with established AI ethics principles.
Liuhuo Wan, Zicong Liu, Chuan Yan et al.· Proceedings of the ACM on So...· 0 citations
With the growing adoption of AI-powered coding assistants, organizations and developers are increasingly seeking to optimize their interaction with these tools. Prior research has largely focused on output quality and productivity gains, with limited attention paid to developers'well-being and interaction experiences. This paper presents a developer-centered empirical mixed-methods study to investigate how professional developers engage with Generative AI (GenAI) in their natural work environment. Controlled data collection sessions are combined with natural work periods. Results show that developers are generally satisfied with GenAI, particularly for monotonous, repetitive, and structured tasks, and report perceived efficiency and productivity gains. Copilot interaction type preferences differ by task type and complexity: While both in-code suggestions and chat-based prompting independently improve task efficiency and reduce perceived workload, combining these interaction types within a single task diminishes benefits. We propose a rule-of-thumb for selecting an interaction type based on task characteristics. During development-heavy tasks, results indicate that perceived cognitive load arises from AI interaction, while perceived productivity depends on AI output quality. Participation in this study positively influenced developers'awareness and intentional use of GenAI tools. These findings demonstrate the value of real-world, mixed-methods study designs to understand GenAI tools and developers'experiences with them.
Charlotte Brandebusemeyer, Kerim Zunic, Thomas Zimmermann et al.· 0 citations
AI-assisted programming tools are widely adopted, yet their practical utility is often undermined by undesired suggestions that interrupt developer workflows and cause frustration. While existing research has explored developer-AI interactions when programming qualitatively, a significant gap remains in quantitative analysis of developers’ acceptance of AI-generated code suggestions, partly because the necessary fine-grained interaction data is often proprietary. To bridge this gap, this paper conducts an empirical study using 66,239 industrial developer-AI interactions from a large technology company. We analyze features that are significantly different between accepted code suggestions and rejected ones. We find that accepted suggestions are characterized by significantly higher historical acceptance counts and ratios for both developers and projects, longer generation intervals, shorter preceding code context in the project, and older IDE versions. Based on these findings, we introduce CSAP (Code Suggestion Acceptance Prediction) to predict whether a developer will accept the code suggestion before it is displayed. Our evaluation of CSAP shows that it achieves an accuracy of 0.973 and 0.922 on the imbalanced and balanced datasets, respectively. Compared to a large language model baseline and an in-production industrial filter, CSAP improves the accuracy by 12.6% and 69.5% on the imbalanced dataset, and by 87.0% and 140.1% on the balanced dataset. Our results demonstrate that targeted personalization is a powerful approach for filtering out code suggestions with predicted rejection and reducing developer interruption. To the best of our knowledge, it is the first quantitative study of code suggestion acceptance on large-scale industrial data, and this work also sheds light on an important research direction of AI-assisted programming.
Jing Jiang, Liehao Li, Jinyu Hou et al.· Proceedings of the ACM on So...· 0 citations
The rapid development and popularization of AI-enabled coding agents have meant software engineering students and professionals cannot be assumed to understand their own code, which risks academic integrity and professional accountability. We developed a method called Personalized Probing Puzzles ($p^3$) to evaluate students'understanding of their own code, and tested $p^3$ in a graduate-level cloud computing course. Our pilot study shows that $p^3$ can help identify potential gaps in students'understanding of their own code. The puzzles are automatically generated, asynchronously administered, and finished in minutes. Future work is needed to correlate puzzle results with code understanding and to embed $p^3$ in a professional code review process.
Yufan Zhang, Jaromir Savelka, S. Goldstein et al.· 0 citations
Large language model (LLM)-powered agents have rapidly evolved from code-completion tools into solvers of complex software engineering tasks. As developers collaborate with coding agents over time, their preferences emerge through repeated interactions and can be used to adapt agent behavior to better meet individual developers'needs. Capturing and reusing these preferences may reduce repeated corrections and improve developer-agent collaboration. Agent skills provide a lightweight mechanism for transferring experience without modifying model parameters. However, existing work primarily focuses on task-specific skills, and it remains unclear whether developer-specific skills distilled from interaction histories can generalize to future tasks. We propose a framework for extracting reusable developer preferences from interaction traces. It first generates personalized skills through rule-based bootstrapping and evidence-grounded refinement, and then evaluates them using a reproducible replay framework with an interactive, trajectory-conditioned LLM-based human developer simulator. We conduct an experiment on 206 real-world developer-agent sessions from 13 developers and compare personalized skills against no-skill, generic-skill, and other-user-skill baselines. Personalized skills provide small and inconsistent improvements over the no-skill baseline, whereas generic skills pooled across developers achieve the largest and most consistent gains. Further analysis suggests that personalized skills become more effective when developer preferences appear frequently, particularly when their histories contain multiple examples relevant to future tasks. These findings provide empirical insights into when developer-specific personalization is effective and demonstrate that broadly transferable procedural knowledge can be more robust than developer-specific preference signals.