Identifying the Dimensions, Challenges, Criteria, Indicators, and Requirements of the Scholarly Peer Review Process and Proposing a Framework for the Evaluation of Scientific Works in Iran

Abstract

Peer review is a scholarly and professional duty for which every researcher bears direct responsibility. This responsibility stems from the fact that the scientific development of any field, along with the enhancement of its journals and scholarly resources, is profoundly influenced by the quality of its peer review process. Consequently, this study addresses the critical need to understand the issues and challenges inherent in Iran's peer review system. Key areas of investigation include: various review approaches and models; considerations for evaluating different types of scholarly outputs; specific challenges in the humanities and social sciences; and the practical and ethical requirements of peer review. The ultimate aim is to provide recommendations for policymakers, peer reviewers, and academic journals in Iran. This study is an applied scientometric research conducted using a mixed-methods approach, integrating both qualitative and quantitative techniques. Data collection involved documentary analysis and survey methods. Initially, requisite information was gathered through a review of national and international literature. Subsequently, data were collected via a researcher-developed questionnaire, whose validity and reliability had been previously established. The questionnaire was administered to a purposive sample of key stakeholders in the peer review process. The study's population consisted of 297 purposively selected participants, comprising 81 academic editors (including journal editors-in-chief, university press editors, and research managers), 102 creators (authors and supervisors of postgraduate theses and dissertations), and 114 reviewers of scholarly works. The results indicated that a hybrid approach to peer review enhances its comprehensiveness and rigor, facilitating more accurate decision-making regarding manuscripts. Furthermore, in the humanities and social sciences, the thematic approach to peer review suggests that reviewers prioritize the manuscript's originality, novelty, quality, utility, and impact over its structural elements. Moreover, given the distinct characteristics of peer review for articles, books, research proposals, and postgraduate theses, the evaluation criteria and indicators for each format differ, underscoring the necessity of applying specific approaches and metrics for each type of work. The stakeholder survey revealed a preference for the 'double-blind' model in the humanities, social sciences, and arts, whereas the 'open peer review' model was favored for basic sciences, engineering, health and medical sciences, and life and agricultural sciences. Additionally, the 'post-publication open peer review' model was identified as an emerging approach, currently utilized by some publishing platforms, publishers, and journals, and is expected to gain wider acceptance within the academic community. However, 'cultural barriers' and 'concerns about potential unprofessional follow-ups between reviewers and authors regarding their review-related correspondence' were cited as major challenges to the adoption of open peer review in Iranian journals. Stakeholders also pointed to 'a lack of motivation for scholarly participation in reviewing' and 'the low weight given to peer review activities in the evaluation regulations for researchers and faculty members' as reasons for poor engagement in the peer review process. Simultaneously, 'critical thinking', 'confidentiality and integrity regarding manuscript content', and 'adherence to a scientific and logical framework' were identified as other essential requirements for peer review in the humanities and social sciences. Respondents deemed an ideal number of reviewers to be 3 or 4 for journal articles, 5 or 6 for book manuscripts, and 1 or 2 for final research project reports. They suggested that for a review to be instructive for the author, reviewers should possess a sense of collaboration and dedicate sufficient time and care. Conversely, for the review process to be educational for the reviewer, special attention should be paid to the research findings, methodological details, and innovations. From an ethical standpoint, stakeholders believed that the anonymous peer review model promotes 'specialized justice' within Iran's academic communities. Furthermore, 47% of respondents considered it permissible and ethical to 'make necessary edits to a peer review report after informing the reviewer, should the reviewer be uncooperative'. Additionally, 55% agreed with 'the combined use of artificial intelligence and human reviewers' to detect potential plagiarism in manuscripts. In conclusion, this research presents specific recommendations, strategies, and ethical imperatives for research policymakers and planners, peer reviewers, authors, and journal editors. It is anticipated that attention to these findings will contribute to the enhancement and elevation of the peer review system within Iran's academic community. Keywords: Peer review, Peer review ethics, Peer review challenges, Peer review models, Peer review approach.