EduSkillBench: Measuring the Impact of Agent Skills on Single-Turn Educational Tasks

Abstract

Agent Skills are reusable packages of procedural knowledge that augment large language model (LLM) agents at inference time. Educational agents are a natural target for such augmentation, because high-quality teaching behavior depends not only on factual knowledge but also on procedures for diagnosing misconceptions, sequencing hints, designing assessments, differentiating content, and structuring lessons. Existing educational benchmarks mostly measure whether agents can solve educational tasks; they do not isolate how a matched Skill changes the outcome.We present EduSkillBench, a controlled benchmark that evaluates educational Skills under matched No-Skill versus With-Skill conditions. The v1 release curates 18 public education-oriented Skills from two open repositories, maps them to six EduBench-inspired scenarios, and constructs 54 Skill-aligned tasks with explicit task-specific rubrics, of which 42 are single-turn tasks across 14 Skills and 12 are multi-turn tasks across 4 Skills. This paper reports the single-turn subset, evaluated with OpenCode (an agent harness) and BenchFlow (an orchestration framework). On a 0 to 1 rubric-reward scale, the observed mean reward rises from 0.767 to 0.948 for qwen3.7-plus (+18.1 pp) and from 0.314 to 0.557 for deepseek-v4-flash (+24.3 pp). The gains are broad but non-uniform: 10 of 14 Skills improve for Qwen and 9 of 14 for DeepSeek, with one negative-transfer Skill per model, that is, one Skill whose With-Skill reward is lower than its matched No-Skill baseline.All reported values are single-run point estimates and are presented descriptively. Because each model row is scored by a same-family judge, cross-model comparisons are descriptive rather than causal. EduSkillBench contributes a reproducible benchmark for studying educational Skill efficacy, and its central design limitation, that tasks and rubrics are aligned to the matched Skill, is stated explicitly and made measurable. The multi-turn subset is released as task specifications and is not yet evaluated.