EvoModelKG: Self-Evolving Knowledge Recovery for Open-Source Model Knowledge Graphs

Abstract

Hugging Face hosts a large open-source ecosystem in which models may reuse upstream models, datasets, tools, and licenses. Yet its structured metadata offers only incomplete coverage of evaluation results, code resources, intended uses, and risks, while model-card READMEs often document these facts in natural language. To address this gap, we present EvoModelKG, a knowledge graph covering all 172,549 Hugging Face models with more than 1,000 cumulative downloads in our July 2026 snapshot, together with the self-evolving knowledge extraction framework that constructs it. The framework treats populated structured fields as weak reference labels for evaluating candidate extractors and uses the resulting feedback to evolve extraction prompts and to create or update deterministic processing tools, all while keeping the LLM weights fixed. Every evaluated version is archived, and the best-performing configuration is periodically restored to limit cumulative drift. Applied to the 130,396 README-bearing models, the selected extractor agrees with overlappping nonempty structured values on 87.16% of 347,376 model-attribute records and attains 74.41% F1 on an independent, mannually annotated gold set of 400 READMEs. Analysis of the completed graph reveals contrasting Qwen-Llama derivation topologies, distinct benchmark portfolios for substantiating capability claims, and capability-specific usage-restriction profiles.