How to select peer reviewers: Advice from an expert journal editor
A manuscript’s journey from an editor’s perspective
Many authors do not know exactly what goes on behind the scenes in a journal’s editorial office, from manuscript submission to final decision. In this series of posts, an experienced journal editor tells us more about the editorial structure of a journal as well as its evaluation and decision-making processes. This series sheds light on the factors that are at play during manuscript publication, how editorial decisions are made, and what the sequence of events in the publication process is.
Editor’s Note: This post was originally published in 2014 and has been refreshed for Peer Review Week 2017.
In the previous part of this series, I gave a broad overview of the Associate Editor’s (AE’s) responsibilities, and went on to describe the process of how an AE is assigned and how the AE reads and assesses the manuscript to decide whether to send it for peer review.
In this post, I will describe how, as an AE, I proceed once I decide to send a manuscript out for peer review. My tasks at this stage include selecting potential peer reviewers and sending out review invitations, identifying more reviewers in case the invitations are rejected, repeating the process till an adequate number of reviewers have accepted the invitations. I hope this post enables you to get a glimpse at academic publishing from the perspective of journal editors and their task of reviewer selection.
Picking reviewers
Once I get an overview of a manuscript and decide to send it for peer review, my next step is to identify a set of potential reviewers to invite. I think about the paper as I understand it, and which researchers are active in that area or related areas. I cast my mind over papers I have read, presentations I have seen, and conversations I have had, to identify who is a suitable expert on the topic. There doesn’t have to be an exact match – perhaps the application is unusual, but the potential reviewer might have used similar techniques.
I also draw ideas for reviewers from the paper. Does the paper make extensive reference to some prior work? Does it compare to a method described in a previous paper? Then there is a good chance that I will invite the authors of these papers (assuming that they do not overlap with the authors of the current submission) to perform the review. I may do some speculative searching – are there keywords or problem descriptions from the paper that I can find other papers about online? In particular, I check if I can find papers on similar topics published in the same journal – since I feel the authors of those works owe a review back to the journal.
After brainstorming for a while, I usually have a list of half a dozen potential reviewers. I do some additional research on them to ensure that they are well-placed to help. Before inviting each reviewer, I check their website, if they have one. I look at the titles and venues of their papers, and years in which they have been active in this area, and also descriptions of their current role and activities.
Apart from their expertise, I consider a couple of other points when picking a reviewer. You might be able to draw some tips from these considerations:
Do they have other commitments?
I tend to avoid asking people who indicate that they are the head of a large research group, chair of their department, and active in running a start-up at the same time. Such people tend to be too busy to perform reviewing tasks. Advanced graduate students can be a good fit because they know their focus area very well, they are likely to be interested in becoming peer reviewers, and have very few other pressing demands on their time; however, it is sometimes hard to tell which students are mature enough in their area without a personal recommendation. So the bulk of reviewing falls upon faculty and researchers who don’t appear too busy, or don’t yet realize how busy they are.
I avoid asking EiCs and AEs of any journal to perform a review: they are usually far too occupied with the submissions for their own journal. In particular, I avoid asking an AE from the same journal to assist.
Are they still active in the field?
The editors' curse is to find someone who has worked on some highly related topics, only to discover that their last publication was in 1999. Usually this means that they have left research for another career, retired, or abandoned this area of study. In some cases, I identify a reviewer who would be perfect to help with a paper, only to discover that they are no longer alive!Following this analysis of reviewers, I pick a shortlist of 3 or 4, and start to send out invitations. Some journals typically have a default invitation template describing the expectations. I personalize this invitation, to give some indication of why I have invited the reviewer: for example, because I think the submission relates to their expertise on a topic. My hope is that this personal touch will make them more likely to accept the invitation. The invitation can also indicate if the paper is a resubmission, an invited submission or an extended version of a conference paper.
I might include the submitted manuscript with the invitation. When I am invited to review, I often find it helpful to quickly scan the submission, to determine how relevant it is and how much effort it will be. When suitable, I like to give other reviewers this opportunity. However, I must admit, when a paper seems particularly long and technically dense, I may avoid sending it, for fear of scaring off the potential reviewer.
Dealing with declined invitations
Inevitably, some invitations to review will be met with rejection. Indeed, in my experience about half of responses are negative. This can be for many reasons, of varying validity: the invitee is too busy, does not consider themselves an expert on the subject matter, does not find the paper interesting, or just doesn’t feel like it on the day. A negative response does not annoy me (unless I feel that the paper really was spot-on for the reviewer). What does irk me are two things:
Tardiness – It should not take a long time to respond to a review request. If people are actively at work, I would hope to hear a reply within a couple of days; if traveling or otherwise tied-up, I would still hope to hear within a week or so. It pains me when an invitee sits on a request for weeks, and then declines (possibly only after a reminder). Even when the invitation is accepted after a long pause, this can be a troubling sign, as it indicates that the review itself may be similarly delayed.
Lack of alternative suggestions – My favourite type of response is actually a very fast negative response that comes with a list of suggested alternate reviewers. This means that the invitee has thought about the invitation, understands that they are unable to commit to it, but has considered it enough to come up with a list of others who may be able to help. This is particularly valuable when the area of the submission is less familiar to me. As a reviewer, I suggest alternates when I am unable to assist – unless I really don’t know the topic. As a result, when a subject matter expert declines, I often follow up with an email pushing for some suggested alternate reviewers. I encourage people to feel obliged to provide alternates when declining an invitation.
Selecting additional reviewers
When reviewers decline an invitation, I need to find more reviewers to invite. Sometimes I have some back-ups already picked, or can take advantage of suggestions from those who have declined. I avoid having more than four “active” invitations at one time, in case all reviewers accept: it is redundant to have a large number for one paper. Often though, I need to find some new candidates. This is perhaps the toughest part of the job, as it means further head scratching to come up with good candidates. It is quite dispiriting when a large number of reviewers have declined to review a paper. The worst case is when the paper is quite specialized, and all the natural candidates have been tapped. It is particularly galling when, after prompting for other reviewers, the suggestions consist of candidates who have already declined. At this stage, the AE can feel that the task of finding enough suitable experts to evaluate a paper may be impossible. However, with persistence, enough reviewers will eventually agree.
Achieving acceptance from reviewers
When sufficient reviewers have agreed to review a paper (usually three or four), and dates for the review have been agreed upon, the initial phase of the process is complete. I can sit back, relax, and wait for the reviews to arrive.
The next part of this series will deal with what happens once the reviews come in and how a final decision on the manuscript is reached.
This post is a modified version of the article What does an Associate Editor actually do? originally published on the website of the Association for Computing Machinery, Special Interest Group On Management of Data.This post has been modified and republished with the permission of the author, and is the second part of a series. Stay tuned for the next segment of this series.
Published on: Sep 12, 2017
Comments
You're looking to give wings to your academic career and publication journey. We like that!
Why don't we give you complete access! Create a free account and get unlimited access to all resources & a vibrant researcher community.
Subscribe to Journal Submission & Peer Review
