HiLLM-CD: LLM-Enhanced Hierarchical Cognitive Diagnosis

Abstract

Cognitive diagnosis (CD) aims to infer students' latent proficiencies from their exercise responses. While most existing approaches are optimized primarily for response prediction, such prediction-oriented training can yield less reliable proficiency. Moreover, representing proficiency as independent concept-wise variables overlooks the relations among concepts, often resulting in fragmented and incoherent diagnoses. To address these issues, we represent each student as an explicit concept tree with node-wise proficiencies, enabling coarse-to-fine diagnosis with explicit concept relations. In this work, we propose LLM-Enhanced Hierarchical Cognitive Diagnosis (HiLLM-CD), a tree-structured encoder-decoder framework that uses the concept tree as an interpretable proficiency representation. Specifically, HiLLM-CD first encodes a student's response history into node-wise proficiencies on the concept tree, and then decodes the tree-structured proficiencies to predict responses via a GCN-based cognitive decoder. Moreover, HiLLM-CD employs a multi-agent LLM pipeline to generate exercise-concept links and automatically construct the concept tree from educational text, substantially reducing reliance on expert annotations and making the overall diagnosis pipeline automated; it further uses an LLM teacher to guide the model to learn more accurate proficiencies. We evaluate HiLLM-CD on MOOCCubeX across four subject domains (Computer Science, Mathematics, Physics, and Biology). In fully unannotated settings, HiLLM-CD improves response prediction by 11.1% AUC on average and improves proficiency reliability by 0.597 in Kendall's tau. We further validate its utility on computerized adaptive testing, where HiLLM-CD improves AUC by 11.4% on average. Our code is available at: https://github.com/xyq20011011/HiLLM.