Skip to content

Who Maintains Agent Skills? A Longitudinal Study of Human-Governed, AI-Assisted Skill Maintenance

Sep 2026 · 0 citations · 36 references
Computer Science

TL;DR

This work mine the full commit histories of five public AI-skill repositories, a purposive sample of AI-tooling organizations, and a purposive sample of AI-tooling organizations from October 2025 to June 2026, and codes each edit with pre-registered governance, operation, and trigger-evidence codebooks.

Abstract

Lifelong LLM agents increasingly rely on external skill artifacts as one element for preserving and reusing capabilities over time. These skills (usually portable Markdown files such as SKILL.md) describe when and how to apply a capability and must be corrected, expanded, and consolidated as tools and usage patterns shift over deployment. Recent work seeks to automate skill curation, but it largely evaluates against automated baselines and treats human maintenance as an unmeasured bottleneck. We study that missing process directly. We mine the full commit histories of five public AI-skill repositories, a purposive sample of AI-tooling organizations, covering 873 commits, 143 skill files, and 254 substantive post-creation edits from October 2025 to June 2026. We code each edit with pre-registered governance, operation, and trigger-evidence codebooks. Three findings emerge. First, every substantive edit is authored or merged through a named human account, while 62% carry an AI co-author trailer, with large repository-level variation. Second, these edits are genuine curation: an audited sample shows that most change skill content, and the coded operations are dominated by additions and corrections. Third, a pre-registered rule-likeness axis fails its reliability gate; reliably coding rule-likeness from commit artifacts remains an open measurement problem. We release the corpus, codebooks, mining scripts, and a replay protocol for automated skill curators. For self-evolving agents, public skill maintenance currently looks less like an autonomous pipeline than a human-governed, AI-assisted loop that future curators must measure against and operate within.

View source

Similar papers

#artificial intelligence Review Oct 2026

An Empirical Study of Agent Skills'Downstream Utility

Agent Skills package procedural guidance and resources for reuse, but a relevant Skill does not necessarily improve task performance. Existing studies characterize Skill content and evaluate downstream performance, yet provide limited explanations of how utility depends on content, execution configuration, and multi-Sk...

Yu Cheng, De-Hai Zhao, Zhong-Xin Liu et al. · 0 citations
#artificial intelligence Preprint Oct 2026

One Skill Too Many: How Co-Installed Skills Conflict in Coding Agents

Coding agents are extended with agent skills, directories whose SKILL.md tells the model when and how to perform a task. Because skills come from independent sources (teams, developers, plugins, copied collections), an installed skill can be co-installed with a similar skill doing the same job, and the model picks betw...

Chao-Liang Yan, Zi-Hao Xu, Yue-Kang Li et al. · 1 citation · ⚡1
Preprint Aug 2026

Demystifying Agent Skills: Why They Work-Until They Don't

This work designs a contrastive study that combines controlled quantitative experiments with paired trajectory analysis and consolidates observations into a taxonomy of three high-level categories and twelve skill-use modes, showing that skills work when noisy trajectories become procedural anchors that stabilize execu...

Zhi-Yuan Jiang, Fan Huang, Hanwen Xing et al. · 4 citations · ⚡1
Preprint Aug 2026

Coalition-Aware Skill Reliability for Self-Evolving Agents

Systematic skill-bank audits reveal two recurring reliability failures: coalition pollution, where bank-level gains conceal negative coalition-level skill contributions, and cross-domain utility reversal, where source-beneficial skills reverse their effects after transfer.

Qiyan Zhao, Xiao-Feng Zhang, Bo Liu et al. · 0 citations
Review Aug 2026

Evaluating Skills, Not Just Agents: Agentic Continuous Evaluation of Skills

ACES (Agentic Continuous Evaluation of Skills), a repository-native framework for evaluating skills and product capability packages as executable agent artifacts, is presented.

Christopher Kevin, Narendran Raghavan, J. Puget et al. · 3 citations · ⚡1
Preprint Aug 2026

Signal or Noise? A Benchmark Study of Agent Skills in Web Development

These findings recast a matched Skill as a hypothesis about a particular Skill-project-model triple rather than a portable asset, reframing injection as a per-deployment routing decision and making length-matched controls and per-model audits a minimum standard for Agent-Skill evaluation.

Zi-Yue Yang, Fan Ding · 3 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.