Skip to content
#generative ai Open access

Supplementary data for "Artificial Intelligence Methods for Automating Literature Review Workflows: A Critical Review"

Sep 2026 · Zenodo (CERN European Organization for Nuclear Research)

Abstract

Supplementary data for the critical review "Artificial Intelligence Methods for Automating Literature Review Workflows," which analyses 191 publications from 2016 to 2025 and maps methods, evaluation practices, and validation maturity across seven literature-review workflow phases. This deposit contains the complete study-level analytical record underlying every count reported in the article. It comprises a Supplementary Data workbook with 191 records coded across 42 fields (bibliographic metadata, publication and study-design classification, workflow phases, exact methods and named systems, technique families, preprocessing, training and test data, evaluation metrics, reported results, resource availability, validation tier, and stated challenges); a condensed included-publication ledger (Supplementary Table S1, 191 records); a full-text exclusion ledger with standardized reason groups (Supplementary Table S2, 60 records); a list of reports sought but not retrieved (Supplementary Table S3, 29 records); and the Supplementary Methods and Coding Framework defining eligibility criteria, controlled variables, validation-maturity tiers, metric categories, and counting rules. Three conventions govern how these files reconcile with the totals in the article. The primary phase is mutually exclusive, so phase totals use a denominator of 191. Technique families and metric categories are nonexclusive and include only methods that were applied or evaluated, giving a denominator of 170; percentages in those tables do not sum to 100 percent. Retrieval-augmented generation is recorded as a distinct architecture, and a RAG system using a generative model contributes to both the RAG and the LLM/generative-AI categories. Validation tiers describe the strength of the evaluation design only and are not risk-of-bias or overall quality scores. Records were assembled from two searches, an initial search covering 2016 to March 2024 and an update covering March 2024 to December 2025. One reviewer conducted screening, eligibility assessment, and study-level coding, supported by structured decision rules and automated consistency checks. The files therefore reflect single-reviewer coding rather than independent duplicate coding. This deposit contains no new primary data; all records derive from previously published literature.

View source

Similar papers

#artificial intelligence Conference Open access Apr 2020

ECCOLA - a Method for Implementing Ethically Aligned AI Systems

The method, ECCOLA, is presented, which aims at making the high-level AI ethics principles more practical, making it possible for developers to more easily implement them in practice.

Ville Vakkuri, Kai-Kristian Kemell, P. Abrahamsson · 64 citations · ⚡6
#computer vision Review Apr 2024

AI-powered Code Review with LLMs: Early Results

The goal is to not only refine the accuracy of the LLM-based tool but also to underscore its potential in streamlining the software development lifecycle through proactive code improvement and education.

Z. Rasheed, Malik Abdul Sami, Muhammad Waseem et al. · 62 citations · ⚡3
#computer vision Open access Mar 2024

LLM-based agents for automating the enhancement of user story quality: An early report

The use of large language models to automatically improve the user story quality in Austrian Post Group IT agile teams is explored, with a reference model for an Autonomous LLM-based Agent System developed and implemented at the company.

Zheying Zhang, M. Rayhan, Tomas Herda et al. · 48 citations · ⚡4
#computer vision Review Mar 2024

System for systematic literature review using multiple AI agents: Concept and an empirical evaluation

This paper introduces a novel multi-AI-agent system designed to fully automate SLRs, and demonstrates how it substantially reduces the time and effort traditionally required for SLRs while maintaining comprehensiveness and precision.

Abdul Malik Sami, Z. Rasheed, Kai-Kristian Kemell et al. · 44 citations · ⚡2
#computer vision Feb 2024

Can Large Language Models Serve as Data Analysts? A Multi-Agent Assisted Approach for Qualitative Data Analysis

The proposed LLM-based multi-agent system automates qualitative data analysis process, creating opportunities for researchers and practitioners, and future improvements focus on enhancing multilingual performance and integrating continuous expert feedback.

Z. Rasheed, Muhammad Waseem, Aakash Ahmad et al. · 41 citations

Related blog posts

Microsoft Research Blog Oct 12, 2026

Simulating student learning for better AI tutoring

StudentSim is a research system for evaluating and improving AI tutoring tools before they are tested with learners. It models how individuals may engage with tutors, giving feedback to improve tutor performance across subjects like chess, writing, and mathematics. The post Simulating student learning for better AI tutoring appeared first on Microsoft Research.

GPT-Lab Oct 12, 2026

AI Has Grown Up. Have We?

AI is moving beyond impressive demos. After World Summit AI 2026 in Amsterdam, GPT-Lab's Muhammad Waseem reflects on what organizations need to make AI agents reliable, responsible and genuinely useful in everyday work. The post AI Has Grown Up. Have We? appeared first on GPT-Lab.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.