Skip to content
#artificial intelligence Review Open access

From Benchmark Accuracy to Industrial Evidence: A Registered Systematic Map and Nested Experiment-Level Audit of Validation Credibility in AI-Based Rotating-Machinery Fault Diagnosis

Sep 2026 · Vibration · 0 citations · 60 references
Machine Fault Diagnosis Techniques

Abstract

Artificial intelligence (AI) methods for rotating-machinery diagnosis based on vibration and acoustic signals are often evaluated on benchmark splits that do not establish physical independence or industrial generalization. This registered study combined a PRISMA-aligned systematic map with a nested experiment-level audit of validation credibility. Searches identified 11,916 records, retained 7520 unique records, and placed 5159 reports in the full-text population. The operational 500-record Batch 005 block produced a non-probability 351-report full-text queue. A post-review eligibility-consistency check yielded 97 included studies, 13 exclusions, 241 unresolved reports, and 157 validation experiments. Within this access-influenced subset, 130 experiments (82.8%) remained at A0–A1 physical independence, 27 (17.2%) reached A2–A4, and 41 (26.1%) evaluated B2–B4 domain shifts. Under a narrower leakage definition, positive evidence supported High risk in 19 experiments (12.1%), whereas 93 (59.2%) were Unclear because reporting was insufficient. Seventy-seven experiments reported classification-like numeric headline scores, but metric types were heterogeneous and were stratified rather than pooled. Metadata diagnostics confirmed source-structured selection and access-related differences between resolved and unresolved reports. The Industrial Evidence Matrix provides a structured vocabulary for validation evidence; all proportions describe the nested audit, not the complete literature.

Read PDF

Similar papers

#artificial intelligence Preprint Jul 2024

The Llama 3 Herd of Models

It is found that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks, and performs competitively with the state-of-the-art on image, video, and speech recognition tasks.

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri et al. · 21,082 citations · ⚡3,832
#artificial intelligence Conference Open access Jun 2018

Gibson Env: Real-World Perception for Embodied Agents

This paper investigates developing real-world perception for active agents, proposes Gibson Environment for this purpose, and showcases a set of perceptual tasks learned therein.

F. Xia, Amir Zamir, Zhi-Yang He et al. · 1,069 citations · ⚡112
#artificial intelligence Open access May 2023

Evaluating the Performance of Large Language Models on GAOKAO Benchmark

GAOKAO-Bench is introduced, an intuitive benchmark that employs questions from the Chinese GAOKAO examination as test samples, including both subjective and objective questions that contribute a robust evaluation benchmark for future large language models and offers valuable insights into the advantages and limitations...

Xiaotian Zhang, Chun-yan Li, Yi Zong et al. · 216 citations · ⚡17

KTO: Model Alignment as Prospect Theoretic Optimization

This work proposes a HALO that directly maximizes the utility of generations instead of maximizing the log-likelihood of preferences, and suggests that the best loss depends on the inductive biases most appropriate for a given setting, an oft-overlooked consideration.

Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff et al. · 195 citations · ⚡28
#artificial intelligence Open access Jul 2024

Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval

This work investigates the possibilities of using LLMs in a resume screening setting via a document retrieval framework that simulates job candidate selection and finds that the MTEs are biased, significantly favoring White-associated names in 85% of cases and female-associated names in only 11.1% of cases.

Kyra Wilson, Aylin Caliskan · 131 citations · ⚡8

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.