Exploring LLM-generated hints for students’ Error Recovery in programming
This paper explores how students in Higher Education use LLMs for programming and how learning environments should scaffold such support. We conducted a repeated-measures exploratory field study in two programming courses, a Bachelor’s and a Master’s level course, using a JupyterLab environment with LLM support. Students encountered different support conditions: no support, generic error-type support, and tailored support using traceback and source-code context. We studied the impact of these conditions on error recovery. Tailored support was associated with higher error recovery than both no support and generic support, a benefit that held across course levels rather than varying with expertise. These findings may indicate that LLM-supported programming environments should supply context-sensitive scaffolding, automatically providing the model with the learner’s code and error trace rather than relying on the learner to communicate that context.