Double-blind and transparent peer review systems have advantages and shortcomings

John Butler-Adam is the Editor-in-Chief of the South African Journal of Science, a multidisciplinary journal, and the second oldest journal in South Africa. He also serves as a consultant to the University of Pretoria. In the past, he has worked with the Ford Foundation as the South African Program Officer for Post School and Higher Education; with the Eastern Seaboard Association of Tertiary Institutions as Chief Executive Officer; with the University of Durban-Westville as Deputy Vice Chancellor (Academic); and with the Institute for Social and Economic Research, UDW, as Director. He earned a PhD in Geography, Environment, and Literature from Penn State University. Over the years, John developed a keen interest in issues in South African higher education and research. He is a frequent contributor to The Conversation and also writes thought-provoking editorials in SAJS. John has authored over 75 publications and made over 100 conference presentations. He also has considerable peer review experience and a deep understanding of peer review and the challenges that SAJS reviewers experience.

In the previous part of this interview John spoke about the South African academic system as well as the interesting concept of post truths when applied to scholarly publishing. The discussion now focuses on his experiences as a journal editor. In this segment, John talks about SAJS and his experiences as a journal’s editor-in-chief (E-i-C). This includes an account of his duties as the E-i-C, the challenges he faces when finding reviewers for papers, and how he resolves difficult situations such as conflicting reviewer comments or author misconduct. Read on for a first-hand account of how a journal editor manages the publishing process.

For the benefit of our readers, could you tell us more about SAJS?

The South African Journal of Science (SAJS) was established in 1903 and so is 113 years old – the second oldest journal in South Africa (after the South African Medical Journal, or SAMJ).

It has been published by various organisations over the years – but, most recently, by the Academy of Science of South Africa (ASSAf) – since 2009. SAJS is one of only around 60 multi-disciplinary journals in the world (and the only one in South Africa) and is ranked at 30 amongst those 52. [Multi-disciplinary journals include Nature and Science.]

As a multi-disciplinary journal, we publish articles that are original, scientifically rigorous and of interest/relevance to researchers and other readers from a range of disciplines.

Submissions are received primarily from South Africa, but also from most other African (all – not just sub-Saharan African) countries and from China, India, Pakistan, Turkey, Greece, the UK and the USA – with a few from elsewhere in the world.

The journal is flourishing – we receive about 900 pre- and formal- submissions (through ScholarOne) each year. Our rejection rate, as of 2016, is 77%. The SAJS now accepts submissions from “pre-print” journal sites, as do many respected journals.

Since 2009, SAJS has been an Open Access journal – free to read, and without charges for submission/publication. It is listed in both Web of Science and Scopus, and is a member of SciELO.

Because SAJS is a multi-disciplinary journal, we have a team of ten specialist Associate Editors who are experts in their areas of research. The “core” team of the Journal consists of an Editor-in-Chief, a Managing Editor, and an Online Publishing Administrator. Since 2016, we have also been a solely “Online” journal, with two free access sites (http://www.sajs.co.za/ and https://issuu.com/sajs) and four online publishing formats.

The Journal is directed by an independent, autonomist Editor-in-Chief (E-i-C) and is overseen by an Editorial Advisory Board whose members serve specified terms of office and who are appointed by the Council of ASSAf. The E-i-C is accountable to the journal’s editorial board.

SAJS has two distinct sections: a “front” section — that carries a Leader (usually written by the E-i-C), News and Views items, Commentaries, Scientific Correspondence, Book Reviews and similar items — and a research section that carries reviewed research articles, letters and research reviews.

What are some of the primary duties you need to perform as E-i-C?

The E-i-C is the “head” or “leader” of the Journal and her/his accountabilities are the scholarly content and quality of the Journal; the timely production and dissemination of the Journal; the effective and timely flow of MSS through their various stages; the leadership of the Journal’s editorial and production teams; and the representation and promotion of the Journal all with the support of the members of the “core” team [Quite a wide and demanding range of responsibilities – if anything goes wrong, the E-i-C is always the first to be in the line of fire!]

And what are some of the most challenging aspects of your work?

Let me start with the real benefits of being E-i-C: I have a wonderful operational team (the Managing Editor and the Online Publishing Administrator) and scientific team (the ten Associate Editors—fifty percent of whom are, by the way, women scholars/scientists; when I started this job, only one of the ten AEs was a woman and we’ve worked really hard to change this).

And now, I’ll talk about some of the challenges:

• Finding reviewers (see below) and ensuring that that they deliver their reviews on time
• Dealing, politely and courteously, with difficult (sometime very difficult) authors
• Ensuring that the deadlines for my many tasks are met or, preferably, attended to ahead of the deadlines (many E-i-Cs do this on a part-time basis)
• Identifying and dealing with plagiarism (this is slowly but steadily increasing). Our policy is (correctly) complex and demanding, and a case of plagiarism can take up to six months and scores of letters to resolve.

Have you faced any difficulty finding peer reviewers for your journal? If yes, how have you dealt with the situation?

Yes. In this respect, we face  difficulties, as do many journals, in (1) identifying suitable reviewers; (2) hoping that they will agree to undertake the review; and (3) ensuring that they submit their reviews on time (although ScholarOne automatically sends regular reminders). We subscribe to the ScholarOne service that provides lists of possible reviewers – and this works at times, but not always.

How do you resolve conflicting peer review comments?

Several options: The AE makes a decision based on her/his expertise; an additional reviewer/reviewers is/are asked to provide a “deal-breaking” review; the conflict is referred to the E-i-C who decides how to deal with the problem. If the negative review is really damning, then the paper is rejected. This final option is very rarely invoked.

And how do you deal with situations when you find that an author has (perhaps even unintentionally) indulged in research misconduct?

An author (or authors) who have breached any of the conditions of our publishing agreement is/are subject to a process relevant to the nature of the breach. If, for example, a paper is being considered by another journal, then we (1) reject the submission and (2) blacklist the author; or, if plagiarism is detected (we use iThenticate for every paper before it is sent to an AE) then the plagiarism policy is set in motion unless we believe that the author is young and inexperienced – in which case we initiate a discussion with the author. All authors who are definitively identified as having breached their contract are black-listed.

Do you think journals can play a role in encouraging authors to follow ethical practices? If yes, how so?

I believe a journal can take several steps to covey its staunch support of ethical research and publication practices. However, this can be slightly tricky and complex. Journals could play their part by always checking articles through iThenticate (or similar system) and perusing plagiarism policies; reporting on numbers of plagiarized papers identified’ and so on. As it happens, ASSAf has published a National Code of Best Practice for South African Journals. Such guidelines are meant to encourage authors to be ethical at all stages of research and its publication.

In terms of frequency, what are some of the most common errors you see authors making when they submit their papers to a journal? How would you advise them to avoid making these mistakes?

Oh dear, Jayashree, this is could be a paper all of its own!  But the simple answer has two elements: (1) authors who submit their manuscripts through ScholarOne Manuscripts directly to the Journal without reading our Guidelines some of which I immediately reject; and (2) authors who read, at least, the part of the guidelines that specify that all manuscripts should first be submitted directly to me (pre-submissions) before following the formal route but who ignore the remaining requirements. In both cases – authors haven’t read our Guidelines, or, at least, the important parts of them.

So, simply – problem one is that authors don’t read the Guidelines. This wastes both their, and my, time. And it implies that they’re either lazy or negligent.

But of all these errors, the most common is that, not having read the Guidelines (or at least not properly) they use the wrong referencing system. In these cases, I simply send the manuscripts back and ask that the authors read the Guidelines carefully!

The second most common error: authors don’t have their English proofread. Those manuscripts also go back with a request that they sort the language out. That problem is so common that we have a standard Manuscript Central (MC) response for this!

Let’s come back to peer review. SAJS is a fully open access journal that follows a blinded peer review process. What are your views about transparent peer review? Do you feel it could be more effective compared to blinded reviews? Will SAJS adopt transparent peer review in the foreseeable future?

Double-blinded review processes have considerable support – and advantages. They allow reviewers to be honest, and they allow authors to feel that they will not be prejudiced by reviewers who might not like them (the authors) personally or on the basis of some scientific (or other) differences of opinion. In small academic communities, though, it is often the case that reviewers (unless they come from a larger pool of scientists) are able to make pretty good guesses as to the identity of an author, inferred from the research area of the paper and/or the language style used by the author. So it is not a fail-proof system, even at an international level. In addition, double-blind systems can be open to abuse – most notably when reviewers, hiding behind their anonymity, write reviews that are abusive, sexist or racist. Transparent peer review helps to avoid these unacceptable practices (although not the thinking that lies behind them, of course). 

The reality is that both systems have advantages, both have shortcomings (including failing to pick up serious problems in papers) while reviews nonetheless remain critical to scientific integrity. The SAJS tries to ameliorate the worst of the faults of the double-blind system by allowing authors to identify reviewers who should not, for good reason, be invited to review a paper. The Journal has not yet contemplated changing the double blind review system for an alternative approach to reviewing. Accepting papers that have been pre-published on discipline-based pre-publication sites (as we now do) does go some way towards ensuring that submitted papers are more likely to be more rigorous than otherwise.

Thanks for the great conversation, John! Your views on peer review practices as well as how journals deal with author midsonduct were really interesting.

Read part 1 of this conversation: Global warming is a reality, not a trick to hamper industrial growth elsewhere in the world