Jul 2026· Annual Conference on Innovation and Technology in Computer Science Education· 0 citations· 18 references
Computer Science
TL;DR
Socratic AI, a VS Code-integrated tutor that addresses this through pedagogically-grounded Socratic dialogue constrained to withhold direct solutions is presented, and evidence that stateful tracking enables adaptive Socratic dialogue that scaffolds productive struggle rather than short-circuiting learning is contributed.
Abstract
Large Language Models in programming education face a fundamental tension: while capable of generating correct solutions instantly, doing so undermines the struggle necessary for conceptual learning. We present Socratic AI, a VS Code-integrated tutor that addresses this through pedagogically-grounded Socratic dialogue constrained to withhold direct solutions. Our core innovation is a stateful misconception detection mechanism that tracks student understanding across dialogue turns. Operating on a taxonomy of foundational programming errors (off-by-one, mutation vs. reassignment, return vs. print, async/parallel confusion), the system evaluates each student utterance to produce per-misconception verdicts (reinforced, weakened, introduced, absent) with certainty scores. These accumulate through small, incremental confidence score updates until a derived resolution threshold is achieved. This enables the tutor to persist on specific conceptual gaps rather than abandoning them when students deflect or express frustration. This stateful tracking enables two key capabilities: (i) the system adapts questioning intensity based on detected learner confidence signals, shifting from diagnostic probes to conceptual contrasts, and (ii) enforces purely Socratic constraints through multi-layered validation that blocks code generation, direct explanations, and multi-question responses. The system is implemented as a VS Code extension with Gemini-based classification back-end and cloud database persistence for longitudinal analysis. In a preliminary deployment with 48 students, we observed increased question specificity, frequent learner articulation consistent with mental model revision, and reduced recurrence of targeted errors on similar tasks. We contribute a deployable architecture, a misconception taxonomy, and evidence that stateful tracking enables adaptive Socratic dialogue that scaffolds productive struggle rather than short-circuiting learning.
A simulation-based textual analysis of prompt design evaluates a frontier large language model as a tutor across 60 scripted sessions on a single topic and point to dynamic, dialogue-aware prompting alongside explicit SRL scaffolding.
Kendall Hartley, Fabiola Sáez-Delgado, Javier Mella-Norambuena· Future Internet· 0 citations
SocraticTrap-CS is introduced, a publicly available benchmark that probes the capacity of open-weight LLMs to generate strategic misconceptions on demand and reframes the evaluation of educational LLMs around pedagogical trustworthiness rather than factual correctness alone.
Marijela Miličević, Mia Rovis, Ratomir Karlović et al.· Information· 0 citations
HeuristicEdu is presented, a two-phase pipeline that aligns Qwen2.5-7B toward Socratic tutoring via supervised warm-up and Group Relative Policy Optimization (GRPO), and Scaffolding Effectiveness and Conversation Depth are introduced to evaluate outcomes beyond surface fluency.
Xiaokun Wang, Siyu Song, Wentao Liu et al.· arXiv.org· 0 citations
Mastering loops and repetition is a well-documented source of difficulty for novice programmers in upper primary school, who often “unroll” a repeated action into a long linear stack of identical blocks rather than recognising the repeating pattern and expressing it with a single repeat block. This study reports a between-groups experiment. We examined whether a multimodal artificial intelligence (AI) assistant tutor — one that interprets screenshots of pupils’ Scratch code and replies with Socratic scaffolding rather than with corrections — improves learning outcomes and engagement relative to traditional classroom lesson. Forty pupils aged 11 to 12 were assigned to two intact classes of 20. The first received a standard, teacher-led lesson on loops, in pairs at shared computers using scratch.mit.edu. The second covered the same material; whenever a pair’s script did not behave as intended, the pair captured a screenshot (a “print screen”) of their blocks and submitted it to the multimodal AI assistant tutor, which read the code and replied with a guiding question that prompted the pair to locate and repair the fault themselves. Pairs followed the driver–navigator pair-programming technique on both groups. We measured learning with a matched pre- and post-test on loops and gauged engagement through a short attitude questionnaire and classroom observation. The tutor group scored significantly higher on the post-test and reported markedly greater interest and enjoyment than the traditionally taught group. These findings suggest that multimodal AI tools that read learner code and scaffold through questioning can enhance both achievement and motivation in introductory programming.
Dimitrios Kravvaris· European Journal of Contempo...· 0 citations
This paper focuses on feedback for open-ended short answer questions in introductory programming, with the goal of nudging students toward success on reattempts without revealing the correct answer, and develops a five-criteria rubric grounded in educational literature for evaluating feedback quality.
Binglin Chen, Rajarshi Haldar, Max Fowler et al.· 0 citations
Large language models (LLMs) often exhibit sycophancy, optimizing for agreement over productive challenge, which severely limits their utility in domains like professional skills training, where growth requires pushback. We introduce, ConvoDojo, a novel conversational AI platform for practicing difficult workplace conversations, engineered not merely as a commercial training application but also as a flexible, instrumented research platform for evaluating conversational AI strategies. ConvoDojo repurposes LLMs as structured sparring partners to support skill development in difficult workplace conversations (e.g., performance feedback, conflict resolution), addressing the reported managerial tendency to avoid them. This paper showcases the platform and presents an evaluation of how key conversational user interface (CUI) design elements, namely, the addition of structured feedback and upfront instructional scaffolding, impact managers’ learning. Results show that ConvoDojo is highly engaging and promotes user reflection. We demonstrate how theory-informed dialogue and adaptive pushback can transform an LLM into an effective, measurable tool for complex communication skills development.
Everlyne Kimani Cross, Luiza A Santos, Laurent Denoue et al.· International Conference on...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
