Oct 2026· Zenodo (CERN European Organization for Nuclear Research)
Software Engineering Research
Abstract
This repository provides the bibliographic dataset and methodological documentation supporting the research article *"Triaxial Evaluation of Large Language Models in Code Generation: A Structural and Semantic Approach"*. The dataset consists of a comprehensive BibTeX (`.bib`) file containing a hybrid corpus of scientific literature focused on the evaluation of AI-generated code, specifically regarding structural complexity, semantic correctness, and the application of Large Language Models (LLMs). The bibliographic collection was constructed through a dual-methodology approach: 1. **Automated Search (IEEE Xplore):** Executed between April 6 and April 20, 2026, targeting metadata fields (Abstract and Document Title) for key benchmarks such as "HumanEval", "SWE-bench", "LiveCodeBench", as well as queries intersecting LLMs, evaluation metrics, and coding.2. **Manual Curation:** Integration of state-of-the-art preprints (ArXiv) concerning LLM-as-a-judge evaluation paradigms, alongside seminal software engineering literature detailing classical complexity metrics (e.g., McCabe's Cyclomatic Complexity, Halstead's Software Science). Essential technical documentation and official repositories for the evaluated tools are also included. The provided `.bib` file represents the raw, unfiltered extraction prior to the application of the research's strict exclusion criteria, which subsequently isolated papers exclusively addressing metrics for AI-generated code. A `README.md` file is included in this repository, detailing the exact search strings, temporal scope, and the exclusion methodology required to reproduce the literature review process.
The results are packaged in the Greenfield Startup Model (GSM), which explains the priority of startups to release the product as quickly as possible, and the need to shorten time-to-market, by speeding up the development through low-precision engineering activities.
Carmine Giardino, Nicolò Paternoster, M. Unterkalmsteiner et al.· IEEE Transactions on Softwar...· 178 citations· ⚡14
Software startup companies develop innovative, software-intensive products within limited timeframes and with few resources, searching for sustainable and scalable business models.
M. Unterkalmsteiner, P. Abrahamsson, Xiaofeng Wang et al.· e-Informatica Software Engin...· 157 citations· ⚡17
This study conducts a case survey study based on the secondary data of the major pivots happened in 49 software startups, and demonstrates that customer need pivot is the most common among all pivot types.
Sohaib Shahid Bajwa, Xiaofeng Wang, Anh Nguyen-Duc et al.· Empirical Software Engineeri...· 127 citations· ⚡15
The comparison of adopter and non-adopter sample reveals three potential adoption inhibitor, security, data privacy, and portability, which underlines the importance of the technical and security perspectives for research investigating the adoption of technology.
Nattakarn Phaphoom, Xiaofeng Wang, S. Samuel et al.· Journal of Systems and Softw...· 111 citations· ⚡8
The ongoing work building a Raspberry Pi cluster consisting of 300 nodes is presented, with potential use cases being an inexpensive and green test bed for cloud computing research and a robust and mobile data center for operating in adverse environments.
P. Abrahamsson, S. Helmer, Nattakarn Phaphoom et al.· IEEE International Conferenc...· 110 citations· ⚡7
The results indicate that software developers are a slightly happy population, but the need for limiting the unhappiness of developers remains, and 219 factors representing causes of unhappiness while developing software are identified.
D. Graziotin, Fabian Fagerholm, Xiaofeng Wang et al.· International Conference on...· 84 citations· ⚡6
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduOct 8, 2026