It is proposed that a quantifiable metric, the Conceptual Difficulty Gap (CDG), may be useful for identifying a class of texts that syntactically appear to be simple, but consistently trigger performance failures.
Abstract
This exploratory study investigates the relationship between student metacognition, use of artificial intelligence, and empirical performance within a multidisciplinary course on AI. Using data from a sample of college-age, full-time undergraduate students (averaging 18 participants per assessment) enrolled in an in-person junior seminar at a Midwestern U.S. university, we correlate student self-assessments with standard readability metrics (e.g., Flesch–Kincaid), L2SCA metrics, and objective assessment outcomes, analyzing how learners evaluate their own comprehension and how they deploy AI tools in response to the complexity of 21 reading assignments over 8 weeks. We find that students’ perceptions of linguistic difficulty correlate with classical readability scores, but their perceptions do not predict their success nor does their engagement with assistive AI. The results suggest that students utilize generative AI tools as a habitual baseline rather than a strategic response to difficult material. We argue that while students can identify surface-level linguistic friction, they fail to recognize deep conceptual hurdles, leading to a false sense of mastery that neither their intuition nor their AI assistants appear to mitigate. We propose that a quantifiable metric, the Conceptual Difficulty Gap (CDG), may be useful for identifying a class of texts that syntactically appear to be simple, but consistently trigger performance failures. Crucially, we uncover a possible metacognitive blind spot: student self-ratings of difficulty are negatively correlated with this gap, implying that student assessments of difficulty are not based on actual conceptual difficulty. Furthermore, self-reported AI reliance shows no correlation with the gap, indicating that students may not be strategically deploying generative AI tools to mitigate conceptual difficulty.
This study examined how Moodle-based continuous assessment tasks associate with undergraduates’ metacognitive engagement in English for Specific Purposes (ESP) reading in a first-year engineering course at a Sri Lankan public university. It used a single-case study design, treating a public university’s ESP reading module as the bounded case. Quantitative (CA1: N = 997; CA2: N = 973) and qualitative (n = 51) evidence were drawn from records of two compulsory continuous assessments and volunteer subset post-task reflections, respectively. The research questions focused on performance patterns across comprehension, summary writing, multiple-choice, and inference tasks; relationships among task components; comparative difficulty and ceiling effects across CA1 and CA2; reported strategy use and online assessment experience; and implications for task design. The objectives were to: (1) analyse student performance patterns across task types in CA1 and CA2; (2) examine relationships among assessment components; (3) evaluate difficulty, discrimination, and ceiling effects; (4) explore learner reflections on strategy use, confidence, and online assessment experience; and (5) propose assessment design implications for capturing metacognitive engagement. Quantitative analysis, using descriptive statistics, Pearson correlations, score distributions, and Moodle quiz statistics, examined difficulty and discrimination. Qualitative analysis, using thematic analysis, identified reported strategies, challenges, and suggestions. Results showed consistently high achievement (x̄ > 97/100) and strong ceiling effects, limiting discrimination among learners. Summary writing functions as a distinct component, showing minimal association with recognition-based items and eliciting evidence of planning, monitoring, and evaluation in reflections. Learners valued summarising and note-taking, reported weaker planning and monitoring, and noted tensions between high scores and low confidence, alongside technical issues during Moodle delivery. Findings suggest rebalancing ESP reading assessment towards more demanding tasks, strengthening productive components, and incorporating brief reflection prompts to capture metacognitive engagement in large-cohort digital contexts.
Ashmini Kalika Karunarathne· Journal of Innovative Practi...· 0 citations
There is ongoing academic debate on whether one can teach AI literacy to undergraduate students across majors, and if yes, how. This article reports a case study: a three-week midterm project embedded in an undergraduate “AI-for-all” course. Students designed reasoning tasks, ran controlled comparisons across widely used chatbots, and evaluated both answer correctness and explanation validity. Through field experience, students with no STEM background learned what consumer chatbots can and cannot do, documenting systematic brittleness across models that “sounded right but reasoned wrong.” More critically, students built understanding of how to evaluate AI outputs. The midterm gave them agency as investigators rather than passive users. Eager to share their discoveries, they are co-authors of this article. Together, we offer here to educators and the broader scientific community a concrete example of the operationalization of AI literacy as experimental practice. The method, however, is not specific to the classroom. It shows any user how to test an AI system rather than trust it blindly. In three-week midterm project, students investigated whether AI literacy can be taught to undergrads.
Amarda Shehu, Adonyas Ababu, Asma Akbary et al.· Communications of the ACM· 0 citations
Mathematics assessment questions influence students' perceptions of mathematics and their capacity for lifelong learning. This position paper argues that test items should be relevant to students' cognitive level, cognitively engaging enough to promote deep thinking, and sufficiently clear to avoid unnecessary confusion. Using the associative and commutative properties as a case study, the paper analyses how the item (x+y)(2m-n) - 2(x+y) can create avoidable cognitive load when students must infer implicit regrouping and subtraction as addition of a negative. Pilot classroom data from 62 SHS 2 students indicated that 71% rated the wording as confusing, 64% reported knowing the mathematics but becoming stuck during problem setup, and statistical tests confirmed that perceived confusion was systematic, with a large effect size (d = 1.08). Qualitative responses showed that students struggled mainly with commutativity, subtraction patterns, and mismatch between the prompt and previously learned examples. The paper presents five confusingversus-clear algebra and geometry examples, integrates classroom evidence with relevant assessment-design literature, and provides a ten-point clarity checklist for teachers. The findings suggest that clear wording, explicit notation, and appropriate scaffolding help students demonstrate mathematical understanding more validly and support the development of reasoning, transfer, and lifelong learning habits.
Dennis Offei Kwakye, Daniel Kudjo Adiku, Alex Boadu et al.· Asian Research Journal of Ma...· 0 citations
Artificial intelligence (AI) offers new opportunities to support individual goal setting and self-regulated learning (SRL). However, its influence on learners’ cognitive effort remains a critical issue. While AI-supported environments can scaffold difficult and complex academic tasks, they may also encourage learners to rely on intuitive, automatic, or heuristic problem-solving strategies rather than sustained analytical engagement. This case study examines how doctoral students in educational technology and distance education in Türkiye use AI during difficult and complex academic tasks through the lens of Cognitive Miser Theory (CMT). Using semi-structured interviews with seven frequent AI users and inductive thematic analysis, the authors identify four interrelated themes: AI’s supportive role in decision-making, effects on cognitive processes, adaptive influence on habit formation, and transformative impact on strategic study approaches. Results indicate that students frequently rely on AI to regulate time, stress, and workload, thereby reducing perceived cognitive effort. However, students with higher metacognitive awareness engage with AI more critically, which, paradoxically, increases their perceived cognitive load. In contrast, under conditions of uncertainty, unfamiliarity, or time pressure, participants tend to rely on AI more uncritically, raising concerns regarding academic depth, learner autonomy, emotional well-being, and epistemic responsibility. The study concludes that AI use in SRL functions as a double-edged cognitive tool: it can either mediate cognitive efficiency or foster cognitive complacency, depending on learners’ strategic orientations and metacognitive capacities. The findings underscore the critical importance of strengthening AI literacy, metacognitive regulation, and calibrated cognitive load management in advanced academic learning contexts. Future studies should incorporate one-on-one sessions with participants, using think-aloud protocols, to achieve a deeper understanding of the topic under examination.
Sehla Ertan, Sezin Eşfer· Open Praxis· 0 citations
Students with similar overall test scores may nevertheless succeed or struggle with very different kinds of problems. Aggregate performance measures can conceal these differences when assessment problems require different forms of reasoning or different levels of structural complexity. Existing cognitive theories and educational taxonomies describe important aspects of knowledge, reasoning, and complexity, but were not designed to classify individual problems simultaneously according to these two characteristics. This study therefore develops a purpose-built Problem-Level Cognitive Framework that represents assessment problems along two analytically independent dimensions: the dominant reasoning required for a conceptually adequate solution—Formal, Procedural, Theoretical, or Critical—and the structural complexity of the minimal valid solution path, expressed through seven ordinal difficulty levels (D1–D7). The framework was empirically examined using semester-long data from 121 undergraduates enrolled in one of two instructional conditions: AI-supported or teacher-mediated introductory statistics instruction. Binary outcomes for each student–problem pair were analyzed using generalized linear mixed-effects models with crossed random effects for students and problems. Successful problem solving varied substantially across reasoning dimensions, difficulty levels, and assessment timings. In contrast, instructional condition made little explanatory contribution, and neither the condition-by-reasoning nor the condition-by-difficulty interaction was significant. The framework thus revealed a stable cognitive organization of performance across two substantially different instructional environments, while also identifying pronounced variation among cognitively differentiated problems. By separating the form of reasoning governing a valid solution from the structural complexity of the required solution process, the framework transforms heterogeneous assessment problems into cognitively comparable analytical units. It provides an operational basis for investigating problem-level performance structures and for examining their stability across learners, assessments, instructional environments, and educational domains.
László Bognár, Peter Horvath, Antal Joós et al.· Education sciences· 0 citations
The present study aimed to determine the level of metacognitive cognizance and reading comprehension of learners in relation to academic performance. One hundred fifteen Grade 11 students were the respondents of the study. Standardized questionnaires were used in this study adopted from Putri et al. (2024). Descriptive correlational research design was employed. The salient results of the study were as follows: the student exhibits a strong, regular awareness of their learning strategies. They frequently plan and monitor their tasks and have a solid grasp of how, when, and why to use specific strategies, though minor inconsistencies may occur in highly complex tasks: the student can read and understand the material completely on their own without teacher assistance; the learners perform very satisfactorily in their academics mastering the core competencies required in their Grade 11 subjects; metacognitive cognizance does not significantly influence the academic performance of the students; reading comprehension is a contributing factor to the academic performance of the students.
Rore Rose E. Janeo-Nunay, Jobell Cris T. Vibal· International journal of res...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
