What Makes Generative AI an Instructional Intervention? A Mixed-Methods Systematic Review of Comparison Conditions, Self-Regulatory Processes, and Independent Performance

Abstract

Generative artificial intelligence (GenAI) is often treated as a single educational intervention even when studies compare different guidance structures and assess outcomes under different levels of tool availability. This mixed-methods systematic-review draft examined comparison conditions, self-regulatory processes, and independent performance in a source-located corpus of 94 full-text records. The 93 reports yielded 94 study units, 80 provisionally eligible studies, 129 arm configurations, 409 outcome points, 1,338 quantitative results, 412 process-evidence records, and 300 qualitative-evidence records. Among 156 performance-classifiable outcomes, GenAI was absent during 41 assessments, present during 31, arm-specific or mixed during 22, and unreported or unclear during 62. A prespecified pooling gate rejected meta-analysis because 103 metrics, five non-equivalent estimand families, sparse compatible cells, and structural dependence prevented a construct-valid pooled effect. Fourteen unique studies measured independent performance: four were favourable, seven null or mixed, two adverse or configuration-dependent, and one a within-GenAI guidance contrast. Monitoring, strategic prompting, and revision were well represented, whereas metacognitive offloading and illusion of understanding were rarely operationalized. Integrated findings suggest that guidance requiring learners to attempt, evaluate, or revise may preserve independent performance more reliably than unrestricted access, but the evidence matrix is dominated by missing assessment-state information. Every primary outcome should report whether eligible generative functions were available, what functions were available, and how restrictions were enforced.