Trust in peer review during COVID-19
“Is peer review broken?” “Are current peer review processes sustainable?” “Is peer review even relevant today?” If you have been following discussions about scholarly publishing over the past several years, then you have already heard these questions dominate many conversations. Nevertheless, despite how flawed peer review in its current form may be, it remains the foundation for trust in the research enterprise.
Most researchers seem to have a good amount of faith in peer review being able to do what it is meant to. Around 98% of the researchers who participated in Publons’ 2018 Global Reviewer Survey said that peer review is either important or extremely important, and about 60% of respondents in Editage’s 2018 global author survey said that peer reviewer comments helped them improve their manuscripts.
If trust in peer review appeared shaky in pre–COVID-19 times, it has certainly been tested severely during COVID-19. Questions are being raised about the effectiveness of peer review under these circumstances, especially with some high-profile retractions of peer-reviewed research papers related to COVID-19.
So, can peer review still be trusted at a time like a global public health emergency? There is currently no evidence to indicate whether researcher perspectives on peer review have changed globally during the pandemic. So, this may not be an easy question to answer yet, but it will be helpful to understand how academia and scholarly publishers have been responding to this crisis.
The major challenge faced by journals publishing COVID-19–related research is to balance the need for speed with the need for rigorous quality checks. Journals have been able to respond to the urgency of the situation well by reducing the manuscript-processing time drastically in spite of burgeoning submissions. Studies have shown that the median submission-to-acceptance time for COVID-19 articles is 6 days, as opposed to a median of 93 days for regular articles in 2019. Moreover, the publication speed for COVID-19 articles seems to be far greater than it was for articles on the 2012 Middle East respiratory syndrome outbreak, which indicates a more concerted effort than earlier by publishers to respond promptly to an urgent public health crisis.
Studies like these usually cannot discuss what editorial strategies were involved in achieving this feat, because it is not always possible to analyze these aspects. So, let’s take a glance at some of the different things that the scholarly publishing community has been doing to maintain the balance between speed and quality.
Journals that have always offered the option of rapid reviews or fast-track publication to authors on request now process COVID-19 articles through this route (e.g., The New England Journal of Medicine). Some adopted process changes to manage the inflow of articles. For example, at the Journal of the American Medical Association (JAMA), COVID-19–related articles that are descriptive and straightforward have been mostly reviewed internally by JAMA editors with relevant expertise, whereas those that are identified as potentially important to shaping practice and policy are reviewed by external reviewers as usual.
However, few peer-reviewed journals have the resources to manage any proportion of COVID-19 submissions entirely by themselves without relying on volunteer reviewers. And not all journals may have the systems that will allow fast-tracking a large number of submissions. This is where larger-level efforts are important.
One such major initiative involves cooperation and coordination between journal publisher groups and between publishers and researchers. In April, a group of publishers, including eLife, Hindawi, PLOS, and Royal Society, issued an open letter of intent for facilitating rapid reviews. This letter called on researchers with relevant expertise to enlist themselves in a cross-publisher shared list of rapid reviews for COVID-19. Reviewers volunteering to be part of this list commit to completing a review within 5 days of accepting the invitation.
COVID-19 is a novel disease that emerged only around 9 months ago, and studying different aspects of this disease has needed expertise from many broad disciplines, from biomedical sciences to social sciences. A substantial amount of time may be lost in a journal’s scramble to just find suitable experts to review each paper and then wait for them to accept review invitations. Also, if the right reviewers cannot be identified or are unavailable, journals will have to rely on secondary reviewers who may not have the required expertise. Therefore, the shared rapid-review list was a much-needed initiative.
The signatories to this open letter also commit to cross-publisher transfer of reviews, so that if a journal believes that a submission will be better suited to a journal run by another publisher, any reviews that may have been performed can be transferred smoothly. This can ensure judicious use of reviewer time and avoid duplication of review effort on a manuscript, especially since reviewers may also become overburdened with COVID-related work.
Furthermore, since preprints have become crucial to rapid dissemination of COVID-19 research, many publishers are encouraging authors to post their articles as preprints. They are also encouraging reviewers to review or flag important COVID-19 preprints on platforms such as Outbreak Science Rapid PREreview.
Efforts like these—that is, those focusing on increasing the efficiency of processes surrounding reviews—are important in ensuring that the actual peer review quality is compromised as little as possible.
However, peer review was never foolproof even in pre-COVID times, so, there have been some slips while journals struggle to maintain this delicate balance. Given the pressures that researchers and journals are experiencing and the implications of flawed research being published, some organizations and authorities have offered guidelines on handling the reviews of COVID-19 articles. The European Association of Science Editors (EASE) issued a statement on quality standards, advising authors and editors to
- Adhere to standard ethical and reporting guidelines like CONSORT and STROBE for epidemiological research
- Share full study data
- Clearly report all limitations related to the study so that readers can make informed decisions based on the results
- Refrain from imposing stringent language requirements as long as a study is understandable and, thus, expedite publication
In addition to quality, transparency is a key element of trust in peer review. The Committee on Publication Ethics (COPE) has urged journals and publishers to disclose any changes made to their editorial and peer review processes because of COVID-19 that may affect the quality of published articles.
COPE also highlights the importance of acting on post-publication feedback that may necessitate corrections or retractions. An example of a journal facilitating this process is JAMA allowing rapid online commenting on COVID-19 papers. This ensures that readers can immediately share any concerns or questions they have, as opposed to using the conventional route of submitting letters to the editor.
The Lancet retracted the now-infamous COVID-19 paper based on data from Surgisphere within two weeks of publication, following serious concerns raised by researchers. It also subsequently modified its peer review process, requiring all authors and reviewers to declare that the data and methods used are reliable.
Opinions on whether peer review can be trusted in these times will vary widely among researchers. Both the benefits and the drawbacks of peer review are likely to stand out in sharp relief. But it is important to appreciate that COVID-19 has presented unprecedented challenges in the past few months and that the scholarly community has been working fast, hard, and together to identify multiple approaches to maintaining the integrity of peer review.
Trust in peer review is a combination of trust in individual researchers and in journals/publishers. Both need to act deliberately and ethically, while responding to concerns in a swift and transparent manner. In a crisis like this, it can be easy to focus more on the lapses, but it is equally important to look pragmatically at how the scholarly community is learning and adapting to the situation and, if possible, to assess the effects of measures taken to safeguard peer review during this prolonged crisis.
This article is powered by R Concept, a platform that combines AI-harvested data with human curation to help researchers working on COVID-19–related topics find the literature and insights they are looking for.
For further insights related to trust in peer review during the pandemic, join us in a panel discussion with experts on Sep 21, 2020. This webinar is part of the Peer Review Week 2020 activities organized by Cactus Communications.
Comments
You're looking to give wings to your academic career and publication journey. We like that!
Why don't we give you complete access! Create a free account and get unlimited access to all resources & a vibrant researcher community.
