What fosters trust in peer review: Perspectives of Chinese researchers
Trust is the bedrock of peer review. Scholarly progress is sustained largely by an institution relying on professionals and volunteer experts to do their job with integrity. The theme for this year’s Peer Review Week—trust in peer review—is timely as well as apt. The scholarly publishing community has been trying multiple approaches to strengthen peer review in the past few years, and the current pandemic has been putting a strain on peer review as never before.
How much do researchers trust peer review? Which factors influence their trust in peer review? How has their trust in peer review weathered the COVID-19 crisis?
In the weeks leading to Peer Review Week 2020, we asked these questions to researchers in China—the leading research producer in the world and one of the countries that has produced the most research related to COVID-19. This post summarizes the results of a survey conducted by Editage.
Survey period: August 20 to September 5, 2020
Total number of respondents: 545
Main questionnaire components: One question asking respondents to rate level of agreement with statements on a scale of 1 to 5 [strongly agree, agree, neither agree nor disagree, disagree, strongly disagree], one question asking them to rank trust-related factors in the order of importance, and one question about effect of COVID-19 on their trust in peer review
Survey dissemination channel: WeChat
Demographic distribution of respondents
The majority of respondents were male, Ph.D. degree holders, and early-career researchers. Most had published over 5 papers, and the largest proportion belonged to the fields of physical sciences, engineering, and related subject areas.
Main results
Around 59% respondents said that they found peer reviewer comments useful, and an overwhelming majority—79%—said that they trust the quality of papers that have undergone peer review more than the quality of those that have not.
(Note: In the figures below, overall agree and overall disagree refer to the total numbers of respondents who chose strongly agree/agree and strongly disagree/disagree, respectively.)
A large majority of the respondents agreed that perception of reviewer bias of any type influences how much they trust peer review and that their trust in peer review would improve if global standards for peer review were established.
We asked participants to rank seven factors based on how important each would be in establishing their trust in a journal’s peer review process. The responses were analyzed by assigning mean ranks to each factor.
The top three most important factors were whether a journal has good reviewer selection policies, whether its review process is transparent, and whether it seeks and implements author feedback on the review process.
Finally, we asked participants how their confidence in peer review was affected by the ongoing COVID-19 situation. The majority—60%—said that it had remained the same as before, and 11%, in fact, believed that it had increased.
Key takeaways
1. Value of peer review. While researchers may be slightly divided about how valuable they find peer review for their own manuscripts, the majority tend to place great importance on peer review in general as a quality-check process in scholarly communication.
2. Fairness and bias. Efforts to improve fairness in peer review by minimizing potential reviewer bias are important. This is underscored by the large majority of respondents saying that perceived bias affects their trust in peer review. Previous surveys have consistently shown that researchers tend to prefer double-blind peer review to other review types. The main reason for this is that the double-blind format is believed to enforce greater objectivity. Approaches to improving reviewer objectivity and accountability are important to establishing trust in peer review, especially among authors from the Global South.
3. Global peer review standards. Establishing global standards can improve not only the quality of but the trust in peer review. It is understandable that peer review processes are determined by many journal-specific factors, and, therefore, peer review quality can vary widely across journals. Nevertheless, perhaps it is time to consider applying standards to at least some aspects of peer review, such as reviewer training programs and expected review outputs. Peer review was already found to be among the top three areas of academic publishing that researchers worldwide want changes in, especially with respect to quality and processes. And defining standards may be even more relevant for urgent, high-impact research, such as research on COVID-19.
4. Trust in peer review through strengthening of processes. The three top-ranking factors that were found to influence trust in peer review were all related to a journal’s review processes and policies. This indicates that even though global standards for peer review may not yet exist, each journal can still improve researcher trust in peer review by adopting robust and transparent processes and by using author feedback to further strengthen peer review.
5. COVID-19 and trust. COVID-19–related pressures may not have extensively damaged trust in peer review among researchers, even though concerns are being voiced about how peer review quality is being maintained and the actual impact of these pressures on peer review is not easy to gauge in the short term.
This article is powered by R Concept, a platform that combines AI-harvested data with human curation to help researchers working on COVID-19–related topics find the literature and insights they are looking for.
For further insights related to trust in peer review during the pandemic, join us in a panel discussion with experts on Sep 21, 2020. This webinar is part of the Peer Review Week 2020 activities organized by Cactus Communications.
Editorial note: This article has been updated since it was first published. An error was discovered in the translation of the rating scale used in the survey question asking respondents to rank factors influencing trust in peer review. We have re-analyzed the results for this question, corrected the graph, and modified sections of the post interpreting those results. We apologize for this error.
Comments
You're looking to give wings to your academic career and publication journey. We like that!
Why don't we give you complete access! Create a free account and get unlimited access to all resources & a vibrant researcher community.
