Generative Artificial Intelligence Use, AI Self-Efficacy, and Higher Order Thinking Skills Among Education Students in Indonesia
TL;DR
The findings suggest that higher education institutions should focus on developing students' AI self-efficacy rather than merely expanding access to GenAI tools, since self-efficacy not access alone is the factor associated with higher-order thinking.
Abstract
Purpose – This study examined the association between generative artificial intelligence (GenAI) use and Higher Order Thinking Skills (HOTS) among education students. It also investigated whether students' confidence in using AI, operationalized as AI self-efficacy, significantly predicts the probability of high HOTS attainment. Design/methods/approach – A quantitative explanatory research design was employed. Data were collected through a five-point Likert-scale questionnaire administered to education students at a single Indonesian university. Binary logistic regression was applied to model the probability of high HOTS attainment, with HOTS dichotomized at the median composite score into two categories: high HOTS (coded 1) and low HOTS (coded 0), using AI use frequency (X1) and AI self-efficacy (X2) as predictors. Findings – The binary logistic regression model (logit(π) = −0.741 − 0.134X1 + 1.891X2) reveals that AI use frequency (X1) is not a statistically significant predictor of high HOTS attainment (Wald = 0.073, p = 0.787). In contrast, AI self-efficacy (X2) emerges as the sole significant predictor, with students reporting higher AI self-efficacy showing significantly greater odds of being classified in the high-HOTS category (OR = 6.623, p = 0.001). With an overall classification accuracy of 73.3% and a balanced accuracy of 61.6%, the Hosmer-Lemeshow goodness-of-fit test confirms adequate model fit, though the model's limited specificity for low-HOTS cases reflects a class imbalance in the sample. Research implications/limitations – The findings suggest that higher education institutions should focus on developing students' AI self-efficacy rather than merely expanding access to GenAI tools, since self-efficacy not access alone is the factor associated with higher-order thinking. Lecturers are encouraged to design learning activities that promote critical and reflective engagement with AI rather than passive use. However, reliance on self-report data introduces the risk of perceptual bias, as students' self-assessed HOTS may not fully reflect actual cognitive performance. Further, because the design is cross-sectional, the findings cannot be used to infer long-term causal relationships. Special attention should also be paid to the generalization of the results as it is narrowed to a limited number of students of education at a single institution. Originality/value – This study is original in its application of binary logistic regression to probabilistically model high HOTS attainment in the Indonesian AI-education context, an approach that has not been widely adopted in prior research in this setting to predict HOTS on the scenario of AI-based higher education in Indonesia.