From psychometric measurement to AI-Assisted interpretation: a generative AI-supported adaptive assessment framework for early numeracy

Abstract

This study aims to develop and validate a Generative AI-supported adaptive assessment framework integrating performance-based assessment, item response theory (IRT) and Generative AI to provide reliable measurement and individualized pedagogical interpretation of early numeracy competence. A psychometric validation was conducted with 210 preschool children aged 4–6 years from six kindergartens in Bandung, Indonesia. Performance on 12 authentic numeracy tasks was analyzed using competing polytomous IRT models, followed by item calibration, differential item functioning (DIF) analysis, latent ability estimation and gender comparisons. The validated psychometric outputs were subsequently interpreted using a standardized Generative AI module. The graded response model provided the best fit to the data, yielding stable and interpretable item parameters, satisfactory measurement precision and reliable latent ability estimates. No significant gender-related DIF was detected, indicating fair measurement across boys and girls. The Generative AI module functioned solely as an interpretation layer, generating consistent narrative feedback and instructional recommendations from validated psychometric evidence. This study proposes a psychometrically grounded adaptive assessment framework that explicitly separates measurement from AI-assisted interpretation, demonstrating how validated IRT-based evidence can be transformed into trustworthy, individualized feedback while preserving measurement validity, fairness and interpretability.