Using Large Language Models to Generate Feedback for Educational Assessments: A Comparative Exploratory Study

Abstract

Feedback in online environments can be an essential learning tool. However, prior research indicates that poorly designed feedback, particularly when punitive or generic, can undermine student motivation and self-perception (Schartel, 2012). In educational assessment, high-quality feedback can provide learners with information about their performance and guidance for subsequent learning. Hattie and Timperley (2007) conceptualized effective feedback around three key questions: (a) Where am I going? (b) How am I doing? and (c) Where do I go next? From this perspective, useful feedback does more than indicate whether a response is correct; it helps learners understand the nature of their performance and identify appropriate next steps. In digital assessment environments, feedback can be tailored to learners' responses and performance. Research suggests that such formative, adaptive feedback can support learning, particularly when it is specific, supportive, and prompts learners to reconsider their performance (Narciss et al., 2014; Shute, 2008). However, manually developing individualized feedback requires substantial time and resources (Leibold & Schwarz, 2015). Multiple-choice assessments offer a useful context for response-specific feedback. Distractors may be designed to represent common errors or misconceptions, allowing feedback to address reasoning plausibly associated with a selected response (Glassey & Bälter, 2020). Response-contingent feedback can provide information beyond simple verification of whether an answer is correct or incorrect by addressing features of the learner's specific response (Shute, 2008). Feedback is also an important component of using multiple-choice assessment for formative learning purposes (Butler, 2018). However, selecting an incorrect response alone does not necessarily reveal the reasoning process that produced it. Consequently, explanations attributed to a particular distractor should be treated cautiously unless the underlying misconception has been independently established. This distinction matters especially when Large Language Models (LLMs) are asked to infer plausible errors associated with distractors.