Exploring the effectiveness of GenAI feedback in facilitating EFL translation task performance

Abstract

Despite the growing potential of generative artificial intelligence (GenAI) to transform language education, empirical evidence on the effectiveness of GenAI feedback in facilitating English as a foreign language (EFL) translation task performance remains limited. Informed by a socio-constructivist perspective that conceptualizes feedback as an ongoing dialogic process (Ajjawi & Boud, 2017), this quasi-experimental study investigated the effects of GenAI feedback on EFL translation task performance, focusing on learners' feedback-seeking behavior, perceptions of GenAI feedback, and immediate learning gains. A total of 121 first-year master's students were assigned to one of three groups: an AI agent group (AIG) who received diagnostic feedback from a customized AI agent and interacted with it, a free interaction group (FIG) who interacted with GenAI using their own prompts, and a teacher-led lecture group (TLG) who received revision instruction through a lecture. Data were collected through two translation tasks, GenAI chat logs, and semi-structured interviews. Results showed that a) the AIG made fewer customization requests and exhibited more behaviors related to monitoring and evaluation than the FIG; b) the AIG reported limited specificity of the diagnostic feedback and questioned GenAI's pedagogical value, while the FIG perceived GenAI feedback as less comprehensible and expressed concerns about overreliance; c) the AIG demonstrated significantly greater immediate learning gains than the TLG, but no significant difference was found between the FIG and the TLG. Findings provide preliminary evidence that GenAI-generated diagnostic feedback can facilitate translation task performance. The study further indicates the need for structured guidance and teacher support to maximize the benefits of GenAI feedback in EFL learning.