13 Takeaways from the 2017 Peer Review Congress in Chicago

I was in Chicago last week, attending the International Congress on Peer Review and Scientific Publication (henceforth, referred to as “the Congress”) along with my colleagues Donald Samulack (President, US Operations, Editage) and Hridey Manghwani (Manager, Corporate Communications, Editage). I must admit that never have I seen such a large congregation of passionate researchers, practitioners, journal editors, reviewers, and other industry professionals intent on discussing one aspect of the cycle of scholarly publishing – peer review.

The Congress was officially scheduled from September 10 to September 12 and the preliminary program included Pre-Congress EQUATOR and Centre of Journalology Workshops on September 9. Typically, after attending a scholarly publishing event, we prepare a report of the sessions and describe what was discussed. But since we have already been sharing detailed daily updates from the Congress through the week, here, I’d like to share with you my observations and thoughts during the sessions.

[Note: These are general observations and reflect my personal views. These should not be considered as a reference to a specific journal/publisher. Also, I had previously read about/formed a view on some of these, but hearing industry professionals articulate and discuss them helped me grasp their scale and urgency.]

1. Peer review is important: No, it is more than important. It is central to scholarly publishing. If you are an author and are not sure about the role peer review plays, the fact that over 500 scholarly publishing professionals across the globe came together to talk about how peer review can be improved for three whole days highlights the significant role this process plays.
    
2. There is a demand for research about peer review: The Congress showcased three days of original research, which included systematic reviews about peer review. Most of the abstracts chosen for presentations and the shortlisted posters focused heavily on empirical, evidence-based research around peer review, how it functions, and what trends are evident based on the research data. The research presented during the sessions explored a variety of topics, e.g., is there a gender bias in a journal’s peer review system, does informing reviewers about open peer review influence their willingness to avoid blinding, how effective are open peer review platforms, etc.
   
   What I took away from this is before we proffer generic opinions such as “peer review is broken,” we need to know that there is a large community of scholarly publishing professionals out there who will not take such statements at face value. They firmly believe that peer review plays a crucial role in the dissemination of good quality research. They also acknowledge that the system has its flaws that can be addressed. The research findings presented at the Congress provided a way to highlight specific issues in peer review and to think of clear cut solutions for improvement.
    
3. You can drive change in the peer review process: The presenters and participants included young scholars, experienced researchers, journal editors, peer reviewers, and publishers. This indicated that the Congress provides a platform for everybody involved in scholarly publishing to undertake and present their research about peer review. For example, one of the research projects presented talked about how making small changes to journal email templates for peer reviewers can lead to critical process improvements. What I assimilated from this is that peer review is not a system that authors are subjected to; they are as much part of it as journal editors and reviewers are.
    
4. Peer review takes time: While hearing journal editors express their views, I noticed that many of them shared a common view. They admit that the conventional peer review process sometimes takes longer, but they believe that there is a justification for this. Peer review should be thorough and rigorous and as such it takes time. Patience in peer review is a value, and if authors believe in peer review, they must trust that it takes some time for referees to put aside their own work and critically assess manuscripts.
    
5. There is little tolerance for unethical practices: Editors are passionate about their journals and realize that they are responsible for publishing research that add value to the field. As such, they want to ensure that they publish genuine research and have little tolerance for unethical research and publishing practices. They feel disappointed when author-initiated malpractices come to light. They want authors to follow all journal guidelines, because in their experience, issues often arise when authors fail to follow instructions and relevant reporting guidelines.
    
6. There is a dearth of reviewers: There is a major imbalance between the volume of journal submissions and the number of reviewers available. Journal editors struggle to find reliable and willing peer reviewers for their journals. This is, in fact, a major pain point for them.
    
7. Peer reviewer training is a hot topic of discussion: Given the discussions around the dearth of reliable peer reviewers, peer reviewer training emerged as a consistent theme during the Congress. The need for training reviewers was acknowledged as one of the ways to hone a future generation of reviewers as well as a potential way to fight the dearth of reviewers. However, there is also some skepticism about assuming that peer reviewer training would directly be related to the quality of reviews. Given the fact that some publishers and organizations (such as Editage, Publons, ACS, and Nature.) are just beginning to offer peer review training, there is not enough empirical data to make an observation in this regard.
    
8. There are mixed views on transparency in peer review: The Congress was attended by many established journal editors with several years of experience managing a journal. This group shared mixed views on transparency in peer review. It was clear that double-blind peer review was associated with greater reliability, but some editors expressed a keen interest in exploring a transparent review process to ensure a better and more equitable distribution of responsibility in the publishing cycle.
    
9. We need process improvements: Journal editors and publishers want to streamline the publishing process. But what they are also looking for is a robust mechanism that will address some of their more common and urgent operational and workflow issues. While they are open to more automation and better tools, they are not keen on blindly incorporating anything that might compromise the quality of published papers. Systematic reviews about different aspects of the journal publishing process, such as those presented at the Congress, help – they are research-based and share analytical data about the gaps in the system and ways by which those could be addressed.
    
10. There is a need to focus on reporting guidelines and best practices: Reporting guidelines and best-practice documents are a critical tool that could help improve the reliability and quality of published research. Journal editors want authors to prepare manuscripts based on relevant reporting guidelines, and they want peer reviewers to refer to these guidelines while assessing a manuscript. This will ensure that all manuscripts are prepared and published in compliance with best practices.
     
11. It's time to accept the hard truth about predatory publishing: Before forming an opinion about authors who publish their papers in predatory journals, it is important to remember that predatory publishers exist because there is a market for them - this hard truth exposes the dark side of academic publishing. And this market exists because everybody involved in scholarly research and publishing contributes to it. Reducing the pressure to publish in high-impact journals, educating authors about how they can avoid falling prey to predatory publishers, and creating a whitelist of journals are some of the ways to help avoid the problem of publishing unreliable research with a bogus publisher. It would also help if, as a community, we understood and acknowledged some universal red flags of predatory journals and shared that knowledge as widely as possible.
     
12. Retraction notices are a sore point: During some of the discussions, it came to light that issuing retraction notices is a sore point. There seems to be an inconsistency in the manner in which retraction related information is issued by publishers. Some journals/publishers include a clarification in the next issue, while some publish a revised paper with a clarification about a previous retraction. The frequency with which these notices are updated is also inconsistent. It also came to light that many publishers are not compliant in posting retractions. These inconsistencies need to be addressed at a global level to avoid the citation loop for retracted research. Having a standard process for tracking and posting retraction notices would certainly help.
     
13. Preprints deserve more attention: The topic of pre-prints generated a lot of interest and debate. The underlying message was that journal editors are not against pre-prints at all. In fact they support pre-prints but would like the scholarly community to exercise caution when considering pre-prints: the problem is that pre-prints are, mistakenly, considered as a proxy for complete, published research. As long as we know that pre-prints are not the same as finished and published research, considering pre-prints should not pose an issue.

As more and more people spoke about the behind-the-scenes aspects of journal publishing, I became aware that this conference attendee group represented the scholarly publishing industry – researchers, medical practitioners, journal editors, reviewers, publishers, industry service providers, analysts – and was passionate about making sure that the best research was published in the most ethical way. I picked up one unarticulated message – it is not enough to simply publish your papers and advance your career. It is important to be aware of the people and processes involved. I found this cog-in-the-wheel realization enlightening because it heightened my awareness of the role Editage Insights plays in bridging the gap between authors and journals.

These are some of my thoughts. If you followed the Peer Review Congress and Peer Review Week, I am sure you might have some opinions of your own. What do you think about journal publishing in general and peer review in particular? I’d love to strike up a discussion with you. Please leave your comments below.

Related reading:

• Biases in scientific publication: A report from the 2017 Peer Review Congress
• Research integrity and misconduct in science: A report from the 2017 Peer Review Congress
• Quality of reporting in scientific publications: A report from the 2017 Peer Review Congress
• Exploring funding and grant reviews: A report from the 2017 Peer Review Congress
• Quality of scientific literature: A report from the 2017 Peer Review Congress
• Editorial and peer-review process innovations: A report from the 2017 Peer Review Congress
• Post-publication issues and peer review: A report from the 2017 Peer Review Congress
• Pre-publication and post-publication issues: A report from the 2017 Peer Review Congress
• A report of posters exhibited at the 2017 Peer Review Congress

Related Infographic

5 Ways statistical specialists supercharge your research

This infographic shows 5 ways in which a…

723

Download the full report of the Peer Review Congress

This post summarizes some of the sessions presented at the Peer Review Congress. For a comprehensive overview, download the full report of the event below.