Peer review in the digital era: Strengths, Weaknesses, Opportunities and Threats

In a recent editorial on Scholarly Kitchen, Roohi Ghosh, Co-Chair of Peer Review Week 2024 has discussed some of the nuances of peer review's role in the digital era. In that article, she has made an ardent appeal to academia to move beyond lip-service and superficial conversations and dig deeper into the issues surrounding the peer review process.

“The aim of AI is to ensure that there is more time for innovation by freeing our time of routine tasks.” – Roohi Ghosh

Considering the increasing presence of AI-generated content in scholarly publications, a deeper analysis was necessary that examines whether the blame for overlooked AI content lies solely with the peer review process. It is also critical that we raise important questions about the current state and future of peer review. In the recent editorial, Roohi has presented the Strengths, Weaknesses, Opportunities and Threats of AI versus the human element in peer review. Although the reader is strongly recommended to go to the original article for the full analysis, you may refer to an abridged version with the main points below:

Strengths:

• Peer review is foundational to scientific integrity.
• Reviewers contribute to research quality by identifying gaps, logical flaws, and improving the overall robustness of studies.
• AI can enhance the peer review process by automating routine tasks
• Reviewers can focus on more substantive critiques -improving efficiency and maintaining high standards.

Weaknesses:

• Reviewers are often over-burdened
• Volunteer nature of peer reviewing leads to fewer peer reviewers
• Lack of diversity among reviewers
• AI-generated content adds complexity to their responsibilities
• Misalignment between reviewer roles and the expectations

Opportunities:

• The conversation around AI in peer review should pivot to opportunity.
• Journals can invest in sophisticated AI tools
• Flag potential issues and streamline the process
• Implementing training programs on effective AI use
• Collaborative peer review approaches
• Establishing standardized guidelines

Threats:

• Over-reliance on AI could undermine the role of human expertise in peer review
• Human judgment may suffer if AI used as a replacement rather than enhancement
• The system risks becoming inefficient if used to detect AI, defeating the purpose

Conclusion

In conclusion, the future of peer review lies not just in identifying AI-generated content but in reimagining the process itself. By recognizing the strengths, weaknesses, opportunities, and threats posed by both human expertise and AI, we can foster a more efficient and effective peer review system. The focus should shift toward collaboration and integration, allowing AI to serve as a valuable tool that supports the core mission of peer review: upholding the integrity and quality of scientific research. Let us engage in a dialogue that truly goes beyond merely scratching the surface of the present and future of peer review.

Related Infographic

The evolution of peer review: A timeline

This Peer Review Week, let’s take a quick tour of some…

1.5k

Roohi Ghosh is the ambassador for researcher success at Cactus Communications (CACTUS). She is passionate about advocating for researchers and amplifying their voices on a global stage.