Investigating the Impact of Student Usage of Generative AI Tools in Computing Courses
TL;DR
The findings suggest that the impact of GenAI use is present in various contexts, highlighting the need for instructional guidance on how students should use GenAI as a learning aid, and insights for other instructors that wish to integrate GenAI tools into computing curricula.
Abstract
Generative Artificial Intelligence (GenAI) tools are increasingly used by computing students, yet their effects on learning outcomes remain mixed. Prior work found that while GenAI use may improve performance on assignments, it can negatively relate to overall course performance. We aim to replicate and extend this work across four computing courses. Using self-reported GenAI usage from assignments and study preferences alongside course performance data, we examine how these relationships vary by course, and compared to the previous study. Our results show that students who used GenAI tools to solve the assignment performed equally or better than those who did not report using it, however, they received lower final grades in the course. We observe no major difference between students who used GenAI to study for the midterm test compared to those who did not. These findings suggest that the impact of GenAI use is present in various contexts, highlighting the need for instructional guidance on how students should use GenAI as a learning aid, and insights for other instructors that wish to integrate GenAI tools into computing curricula.