Human-written and AI-generated revisions: an appraisal analysis of dialogic positioning in argumentative writing

Abstract

This study presents a systematic comparative analysis of appraisal resource use across human-written and AI-generated texts. Drawing on Appraisal Theory, with a focus on Engagement, it compares first drafts, peer-feedback revisions, and AI-generated revisions, which were produced by ChatGPT-4 based on students’ first drafts using a standardized prompt. To conduct the comparison, a corpus of 168 essays across the three draft types was compiled based on four writing tasks completed by undergraduate English majors in China. All drafts were manually coded and data were analyzed using linear mixed models and descriptive statistics. First, AI-generated revisions employed significantly more appraisal resources than peer-feedback revisions, which in turn exceeded first drafts. The increases were primarily driven by Attitude and Graduation resources, while the frequency of Engagement resources remained stable across all three draft types. Second, peer-feedback revisions shifted toward a more contractive dialogic stance, whereas AI-generated revisions maintained a relatively balanced distribution of Contract and Expand within Heteroglossic Engagement. Third, Counter showed a consistent increase in both revision conditions, Endorse rose in peer-feedback revisions but declined in AI-generated revisions, and Pronounce decreased in both revised drafts. The findings suggest that AI-generated revisions may serve as a complementary pedagogical resource rather than a substitute for human feedback, providing students with opportunities to examine different stance-taking strategies in argumentative writing. Given the small participant sample, these findings should be interpreted cautiously. Future research with larger and more diverse samples can examine whether these patterns generalize across different contexts.