Analysis Code and Reproducibility Materials for "Measuring Cognitive Surrender: Early-Warning Signals in AI-Mediated Education"

Abstract

This repository contains the analysis code, derived results, figures, tables, and reproducibility materials supporting the manuscript “Cognitive Surrender in AI-Mediated Education: Behavioral Reliance, Prospective Signals, and Governance for Human Cognitive Agency.” The study examines reliance-oriented behavioral patterns in student-Large Language Model (LLM) interactions using the publicly available StudyChat dataset. It analyzes 16,851 student-LLM interactions from 203 users across 2,214 chat sessions and investigates whether observable interaction traces can characterize patterns of AI-mediated reliance. The repository includes the analysis pipeline used to construct and evaluate the prototype Reliance Index (RI), an exploratory behavioral composite designed to summarize interaction patterns associated with output seeking, limited contextualization, escalation from hints toward direct solutions, repeated prompting, code/data conversion, and limited observable evidence of prior independent effort. The RI is intended as a research instrument, not a diagnostic or psychometric measure of cognitive surrender. The archived analyses include cleaned intent distributions, RI distributions across users and StudyChat topics, comparisons between the RI and an independently constructed Struggle Index, robustness and sensitivity analyses, and a temporally separated prospective analysis examining whether features observed during the first 20% of each user’s interactions contain information associated with reliance-oriented behavior during the subsequent observation period. RI robustness was evaluated using equal weighting, Monte Carlo perturbation of component weights, and leave-one-component-out specifications. Under equal weighting, the RI showed a Spearman rank correlation of 0.953 with the baseline specification. Across 1,000 random ±20% weight perturbations, the median Spearman correlation was approximately 0.995, with a 5th-95th percentile range of approximately 0.980-0.999. Leave-one-component-out analyses indicated that the RI was more sensitive to removal of some behavioral components, particularly low-context prompting and write-code requests, than to moderate changes in numerical weighting. The prospective analysis used temporally non-overlapping predictor and outcome periods. Logistic Regression achieved an AUC of 0.584 and Random Forest an AUC of 0.598 for identifying users in the highest quartile of subsequent-period RI. Feature-set ablation showed that interaction-volume features alone performed approximately at chance, while reliance-oriented and combined behavioral features provided modest prospective discrimination. These results are interpreted as preliminary evidence of temporal structure in interaction behavior rather than as support for individual-level prediction or automated intervention. The repository is organized into directories containing the final analysis code, derived result tables, analysis summaries, reproducibility outputs, and manuscript figures. The included README provides instructions for reproducing the analyses. The original StudyChat dataset is not redistributed in this archive. Users should obtain the dataset from its original repository and place the downloaded data file in the location specified in the README before running the analysis pipeline. Use of the original dataset remains subject to its applicable licensing and usage conditions. This archive accompanies the manuscript submitted for scholarly publication and is intended to support transparency, reproducibility, and independent verification of the reported analyses.