The peer review process: challenges and progress

You have 20 years’ experience as a managing editor, over which time journal processes have evolved remarkably. In your opinion why are some researchers and peer reviewers dissatisfied with the traditional peer review process?

There has been considerable evolution in the way journal editorial work is done, such as the introduction of online working and tools to detect textual duplication or inappropriate image manipulation, but the basic principles of good practice in peer review remain the same – they are independent of system and access or business model. There have always been criticisms of peer review, but they are now more visible and the subject of much public discussion and debate. Also, the scholarly publishing world, and its relationship with the research community, is changing. However, surveys show that researchers generally want to improve peer review rather than get rid of it. The Taylor & Francis Open Access Survey (March 2013) found that peer review was rated the most important service authors expect to receive when paying to have their papers published open access,  coming above both rapid peer review and rapid publication. A major problem is the great variation in the quality of peer review across journals. Also, the ‘fake reviewer’ cases that started emerging in 2012 (details can be found in the ‘faked emails’ category on the Retraction Watch blog) decreased confidence enormously because it became apparent that some journals weren’t carrying out the most basic reviewer checks. Even I, who thought I’d seen it all in my 30-plus years in scholarly publishing, was shocked.

There is also increasing concern and impatience with the ‘wastage’ in traditional peer review, where manuscripts may be reviewed a number of times, going from journal to journal until accepted for publication. This is stretching a system that is already starting to creak. I’m hearing from a number of editors that they are finding it more and more difficult to find reviewers. This is, therefore, one of the main areas people are addressing and looking for solutions, for example cascading review, where manuscripts and the associated reviews can be passed on with the authors’ permission to other journals for consideration, and portable review, where reviews are provided to authors, sometimes by independent peer-review services, and the authors can include them with their submissions. Other initiatives to avoid review and reviewer time wastage include review and referral to best-fit target journals by independent companies.

The internet has opened up opportunities for experimentation with new systems and models of peer review. Many new players are entering the field, some from outside the traditional journal ecosystem. It is a very exciting time! It’s difficult to predict which initiatives will be successful, which sustainable and scalable. They do, however, represent a real challenge to traditional journals and publishers, who need to evolve and innovate, which they are beginning to, to provide what the research community needs and to meet the expectations and requirements of research funders.

Two other issues have contributed to researcher dissatisfaction with peer review. First, the lack of transparency in the process, and efforts are being made to address this, for example by publishing the reviewers’ reports and editorial decisions with articles. Second, there has been growing frustration that peer-review activity hasn’t to date been considered as something that ‘counts’ in a researcher’s research and scholarly contribution record. That too is changing and projects are underway looking into how peer-review activity can be appropriately acknowledged and tied to, for example, researchers’ ORCID records (http://orcid.org/blog/2014/04/08/orcid-and-casrai-acknowledging-peer-review-activities). Once peer-reviewing is awarded appropriate recognition as a professional activity, this may act as an increased incentive to review and provide quality reviews.

In light of the recent cases of unethical or poor research practices, what should the scholarly community realistically expect from a peer review?

Members of the scholarly community have a right to expect that the work they send to journals is treated and assessed appropriately and fairly, also that information about its submission and review is kept confidential unless a journal’s model involves openness, and that no one involved in the handling and peer review of their manuscripts uses that privileged information for their own gain.  They have a right to expect good constructive reviews from individuals who are experts and that it is the work they’ve submitted that is being assessed, not them personally. If researchers get quality service when they are authors and are treated well by journals when they are reviewers – not all are - it’s more likely that they’ll take part in reviewing for those journals and provide quality reviews.

Researchers have many commitments; they are often overstretched, and their time is valuable. Many, quite understandably, feel considerable frustration when they have to spend more time than should be necessary trying to submit manuscripts or reviews because of inadequacies in journals’ system or instructions. All journals should regularly evaluate their processes and make sure they are fit for purpose. All the information and correspondence for authors and reviewers should also be checked regularly and updated as necessary, ensuring consistency across website pages, the online manuscript system, and correspondence. If this isn’t done, researchers may decide to go elsewhere next time to submit or review.

Dr. Irene Hames with Dr. Yan Shuai, President of the Society of China University Journals and Deputy Director of Tsinghua University Press (which publishes the Chinese version of Dr. Hames peer review book, Peer Review and Manuscript Management in Scientific Journals). Photo taken at International Seminar of Medical Journal Editors, Shanghai, China, April 2014, where Dr. Hames gave invited talks on peer review and publication ethics.

You have explained that peer review relies on a “reciprocal process” in which “authors and reviewers are mostly the same community” and everyone does their fair share of reviewing. Is it challenging for journal editors to ensure that there is a good balance between submitting and reviewing manuscripts?

It’s quite straightforward nowadays for journals to monitor reviewing activity at their own journals - manuscript tracking systems make this very easy. A potential reviewer’s reviewing load, present and past, should always be checked before they are approached to make sure they aren’t overloaded at any one time or haven’t been asked too many times over the past year. But editors can only work towards achieving a good balance between submissions and reviewing at their own journals (and give gentle reminders to those who submit lots of manuscripts but do little reviewing for them). Researchers get review requests not only from a range of other journals, but also from grant funders and, now, the new organizations that are entering the field. All are competing for the same pool of reviewers, so building strong relationships and treating reviewers well are becoming even more important.

Be sure to read the last part of this interview series, Keeping the scholarly record sound -- a journal editor's duty. Dr. Hames explains how journals should handle studies with flaws. She also discusses how she works with over 5,000 scientists, from Nobel prize winners to postdoctoral researchers and PhD students to help promote a public understanding of science.