Learning how to perform a review should be part of the PhD training

I am in conversation with Dr. Jonas Ranstam, the medical statistician who was the overall winner of Publons’ Sentinels of Science Awards in 2016. Researchers are busy people, and Dr. Jonas Ranstam is perhaps the busiest of them all. He is, officially, the world’s most prolific peer reviewer, having reviewed as many as 661 papers in a year. He was also acknowledged as one of the Top Reviewers for 2016 by Publons. I feel honored for having this opportunity to talk to Dr. Ranstam about a range of topics from medical statistics to peer review.

Before retiring from being a full-time academic, Dr. Ranstam was affiliated with several institutions, including Lund University, Sweden, as Professor and senior lecturer of medical statistics. Currently, as a medical statistician, Dr. Ranstam acts as a statistical advisor to clinical and epidemiological investigators at academic and research institutions, hospitals, governmental agencies, and private companies. He also offers his expertise to Osteoarthritis and Cartilage (as deputy editor), the British Journal of Surgery (as statistical editor), and Acta Orthopaedica (as a statistics consultant), and a statistical reviewer for several international scientific medical journals. He also maintains the Statistical Mistakes blog, which focuses on systematic reviews of statistical mistakes in medical research and presents references to literature describing how to avoid such mistakes.

In the previous segment, Dr. Ranstam discussed issues with statistical methodology and the irreproducibility of scientific findings. This segment is all about his work as a journal editor and peer reviewer. In addition to talking about his peer review output, Dr. Ranstam talks about review recognition, changing trends in peer review, and peer review scams. He also talks about the need to train peer reviewers at an early stage in their research career and shares some useful tips for them.  

You are officially the world's most prolific peer reviewer, having completed as many as two reviews per day between October 2015 and September 2016. You did this alongside your other commitments – maintaining a blog, consultancy projects, your own writing, teaching, travelling for work, and journal editing. How did you manage to get so much done? Have you ever found yourself wanting to spend less time on a review?

It may seem a lot, but considering that I have more than 20 years’ experience, that I don't commute to work, and that spend very little time on teaching and administration, it was not an unbearable burden. 

It is certainly depressing to see the same mistakes being repeated in manuscript after manuscript, and occasionally, I do get frustrated about the time it takes to explain the same errors and motivate the same corrections over and over again. On the other hand, many manuscripts are well-written, present sound and interesting studies, and are a pleasure to read. The opportunity to suggest changes that can improve such studies further is encouraging and compensating. 

How important do you think it is to reward reviewers? And what is the best recognition a reviewer can get?

Being invited to review a manuscript is, in my opinion, an honour. It is an indication that you are a recognized international expert, and it provides a unique opportunity to influence scientific publishing. Reviewing manuscripts is also a good way to learn about current research topics and to keep ahead of the methodological developments. For a researcher, this is indeed rewarding. The job can hardly be considered thankless. 

Then again, time is limited for most of us, and administration is for many medical researchers an important part of their career. I doubt that reviewing and administration have the same fruitful interactions as reviewing and research and as reviewing and teaching. So from this viewpoint the thankless-job argument may be understandable. 

In an interview, you mentioned that “If it had been possible, I would have made reviewing a more integrated and natural part of the day-to-day activities of active researchers.” How would you have gone about doing this?

I believe that learning how to perform a review should be part of the PhD training, and spending time on reviewing should be encouraged by supervisors and superiors. An increased merit rating for manuscript reviewing would be beneficial. 

How has peer reviewing changed the way you read and understand scientific research?

I am convinced that reviewing changes the understanding of scientific research, and that this change makes you a better author. For example, my own reviewing strategy was initially more focused on isolated technical issues, but with time I developed an understanding of the greater importance of a transparent and consistent presentation, aspects that are as important when writing a manuscript as when reviewing it. 

What do you think about alternative models of peer review, e.g., post-publication peer review, open peer review, etc.?

Pre-publication peer review is undoubtedly important, perhaps especially for the authors. I never submit a manuscript to a journal myself, without first having asked a couple of colleagues to review it, and I am usually very grateful for the additional suggestions I get from the journal reviewers.

Alternative peer review models, such as post-publication review and open peer review are clearly interesting, not least because the conventional pre-publication review systems often lack transparency, which probably is a contributing factor for the attempts to manipulate the system. 

Why do you think peer review scams happen? How can journal editors and publishers prevent peer review manipulation?

Scams happen because they can, and because some people do not mind using unsound shortcuts. To avoid this, editorial awareness is crucial, but a greater transparency of the reviewing procedure can probably also reduce the risk of manipulation. 

As an editor, did you face situations when you found it difficult to find reviewers? What did you do then? How can researchers be encouraged to take on more peer reviews?

Yes, this is a common problem, and I do not have a simple solution. I try myself to grow a personal network of colleagues interested in reviewing, and when I was working at Lund University, I held peer review seminars and organized journal club meetings with review discussions. 

Pre-conference workshops for young investigators at major international conferences could perhaps be useful. I believe that many young investigators would be interested in this. 

Do you think peer reviewers need training? If yes, who should train them and how?

Methodological issues are typically in focus during a review, whether the reviewer is a statistician or a physician, and methodological competence is undoubtedly important. However, I believe that such training is primarily a matter for the reviewer himself or herself as this is a part of the development as a researcher. 

The journal and publisher can perhaps support this development by emphasizing the methodological importance - for example, by endorsing reporting guidelines - and by incorporating technical reviewer support in the editorial system - for example checking guideline compliance or validation of statistical calculations. 

Based on your experiences, do you have any tips for peer reviewers?

Yes, here are some suggestions. 

Do not hesitate to criticize the manuscript, but be constructive and polite. Find the unclear, vague, and ambiguous parts of the manuscript, and the loose ends. Explain the shortcomings to the authors. 

Check if the empirical support for the findings is correctly presented, if the authors' conclusions are consistent with the analysis results. Examine whether the statistical analysis is based on an adequate methodology. Examine if the analyzed data are clearly described, and if they are relevant with regard to the purpose of the study. 

When you detect problems and inconsistencies, explain these to the authors, suggest changes, and motivate your suggestions, but do not be offended if authors decline to comply with your suggestions. 

That brings us to the end of this enlightening conversation with Dr. Jonas Ranstam, experienced medical statistician, senior academic, avid blogger, and the world’s most prolific peer reviewer. Thank you for your time, Dr. Ranstam!