Jul 2026· International Journal of Emerging Technologies in Learning (iJET)· Vol 21· 0 citations· 14 references
TL;DR
A classroom-deployed quiz system that combines integrity-preserving hinting (TA-AI), trace summarization (Analytics-AI), and feedback-driven refinement (AI-Improver) to generate instructor diagnostics from routine interaction logs is presented.
Abstract
AI support is increasingly embedded in online quizzes, yet instructors often lack clear, actionable signals about where students struggle during those assessments. We present a classroom-deployed quiz system that combines integrity-preserving hinting (TA-AI), trace summarization (Analytics-AI), and feedback-driven refinement (AI-Improver) to generate instructor diagnostics from routine interaction logs. The system was used in a graduate assembly programming course over five quiz weeks (N = 18). We report deployment evidence focused on reliability and instructional usefulness for monitoring: promptintent coding reached substantial agreement (Cohen’s kappa [κ] = 0.81); fixed-effects models (with student and item controls) showed a negative association for one-hint interactions (odds ratio [OR] = 0.231, indicating approximately 77% lower odds of a correct response for single-hint interactions relative to 0-hint interactions); and item-level demand spikes were operationalized via a demand × success prioritization process for weekly review. Rather than producing automated judgments or claims of causal learning gains, the analytics are designed as practical prioritization cues that direct instructor attention toward high-need items during AI-assisted quizzes.
Background. Large language models are increasingly deployed as tutors in introductory programming courses, yet evidence that they actually improve learning remains thin, and their tendency to shortcut productive struggle raises concerns about pedagogical harm. Self-regulated learning (SRL) and cognitive engagement (CE) frameworks offer a principled way to address this, but whether embedding them in system prompts actually changes how students learn is an open question. Objectives. We investigated whether AI tutors guided by SRL and CE frameworks affect conceptual understanding, perceived usability, cognitive load, student engagement, and other outcomes when compared to a pedagogically constrained baseline tutor in CS1. Methods. We conducted a preregistered, three-armed crossover study over six weeks of authentic coursework, comparing a baseline AI tutor against two SRL and CE-guided tutors designed to model Zimmerman’s cyclical model, which scaffolds planning, monitoring, and reflection, and Chi’s ICAP framework, which promotes progressively deeper forms of cognitive engagement. We assessed outcomes using post-exercise surveys, conceptual multiple-choice questions, in-platform ratings, and coded responses from the interaction logs, and analyzed the quantitative measures with mixed-effects models. Findings. On the four preregistered confirmatory measures, we found no statistically significant differences between conditions. Non-confirmatory analysis showed that students spent significantly more time on task, wrote longer messages, and produced more constructive contributions when interacting with SRL and CE tutors. Furthermore, the relationship between cognitive load and quiz performance differed significantly by agent type. Implications. Our results suggest that the pedagogical behavior of AI tutors may not be easily steered through system prompts alone: embedding established SRL and CE frameworks did not produce detectable improvements on any preregistered outcome in a large, ecologically valid deployment. Rather than prescribing a single tutoring strategy, future designs may benefit from giving students greater agency over the kind of help they receive, allowing them to choose between scaffolded and more direct support based on their own needs.
Maximilian Georg Barth, Sverrir Thorgeirsson, K. Etemadi et al.· International Computing Educ...· 0 citations
Large language models (LLMs) are increasingly used as on-demand conversational learning assistants, but they typically do not adapt explanations to a student’s background unless explicitly prompted. We present the Personalized Learning Assistant Interface (PLAI), a web-based prototype that generates explanations from lecture slides, audio transcripts, and a structured student profile through a chat-based interface. We evaluated PLAI in a controlled pilot study with 24 STEM students, comparing profile-based personalization with a baseline condition using the same slide and transcript context. Immediate learning was assessed with a five-item knowledge test, while subjective experience was measured using the User Experience Questionnaire (UEQ) and open-ended feedback. We did not observe clear differences in knowledge-test outcomes, but participants in the personalized condition reported significantly higher UEQ Stimulation. These results suggest that profile-based multimodal prompts may mainly support motivational engagement rather than immediate test performance, while larger and longer-term studies are needed to assess learning effects.
Furkan Ali Yurdakul, Yiman Wu, Maria Torres Vega et al.· Message Understanding Confer...· 0 citations
The findings suggest that AI-generated feedback supported targeted revision when it is accessible, interpretable, and aligned with classroom assessment criteria.
A. Tzirides, Michele Galla, B. Cope et al.· Ubiquitous Learning An Inter...· 0 citations
Overall, it can be concluded that schools can realize gains if they couple AI with clear learning goals, teacher capacity-building, and robust measurement plans.
It is found that there is no significant differential effect of GenAI availability on grades overall or among previously lower-performing students, and the findings temper concerns that GenAI inflates grades and reduces students's satisfaction.
James M. Zumel Dumlao, Meng Wang, Zhong-Han Xie et al.· arXiv.org· 0 citations
The results suggest that learner- and curriculum-aware alignment may matter more for effective tutoring than model category alone, and that such alignment is both measurable and improvable.
Benjamin Barlog, Hudson Craig, Ze-Dong Peng· IEEE International Conferenc...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
