From Job Postings to Curriculum Feedback: An LLM-Driven Framework for Constructing Role-Stratified AI Competency Knowledge Graphs

Abstract

AI-related occupations are changing rapidly, whereas higher education curricula typically follow longer institutional revision cycles. Job postings provide timely but unstructured evidence of employer-stated competency demand. Following a design science research approach, this study develops and evaluates a modular framework that transforms AI job postings into a role-stratified competency knowledge graph for curriculum-oriented analysis. We applied the framework to 4,612 job postings from Shenzhen, China, integrating four-dimensional competency extraction, five-stage entity alignment, occupational-role classification, and graph construction. On a 461-record human-annotated reference set, the full Qwen3-Max extraction protocol achieved a precision of 0.898, recall of 0.786, and Micro-F1 of 0.838, with the highest precision and Micro-F1 among the four extraction configurations. On a common stratified review sample of 1,286 manually reviewed entity pairs, the full alignment pipeline achieved a sample-based pairwise cluster precision of 0.960, recall of 0.590, and F1 of 0.731. A separate relaxed-candidate audit found a conditional missed-merge proportion of 6.5% within the relaxed same-dimension candidate space. In a 200-record predicted-label-stratified audit, the equally weighted role-assignment precision was 94.5% (Wilson 95% CI: 90.4%–96.9%). The final graph contains 30,063 nodes and 157,858 relationships and enables descriptive role-level comparisons of employer-stated competency demand. A case analysis using a public talent-cultivation plan demonstrates how graph-derived role profiles can be compared with an existing curriculum to inform program-level review. The study contributes a reusable framework that connects recruitment-derived competency evidence with broader AI education goals, together with a module-level validation strategy that supports transparent assessment and role-informed curriculum review.