Identifying problematic images: Tips and resources for peer reviewers
Inappropriate image manipulation has become increasingly prevalent in research publishing. Unfortunately, because of the evolution of new technologies that allow such manipulation, this problem is likely to persist for some time. For example, artificial intelligence (AI)-generated deepfakes are the recent addition to the expanding types of falsified images1.
One silver bullet to address this huge problem is automated assessment of images for authenticity, but we are not there yet. Of course, there has been some progress on how falsified images are detected, with the introduction of novel methods for image forensics, AI–/machine learning–based tools to detect image duplication2, services provided by data-integrity professionals, etc. But the crucial element in combatting this challenge is still the peer review process.
Role of peer reviewers in upholding research integrity
Peer reviewers are the first line of defense that can prevent doctored images seeping through published literature. However, in several cases, potentially fraudulent images have escaped the scrutiny of peer review because of lack of either vigilance or skill in identifying problems.
One such example is the infamous story of Hwang Woo Suk, a veterinarian researcher who was accused of research misconduct and of manipulating images in a stem cell–research article that was published in a major scientific journal3. Grave concerns were raised about potentially fraudulent data slipping through the cracks. Other examples include the controversies surrounding cancer researcher Min-Jean Yin4 and immunologist Silvia Bulfone-Paus5, several of whose papers were discovered to have included forged images. Thus, peer reviewers bear an important responsibility identifying such cases and upholding research and publication integrity.
Many cases of problematic images are identified by researchers after papers are published. Some of these researchers—who often also serve as peer reviewers otherwise—have driven research-integrity initiatives and set up companies based on their work. Mike Rossner was one of the earliest pioneers of such initiatives6. More recently, biologist Enrico Bucci detected image duplication and other data errors in a homeopathy study published in Scientific Reports7.
For the research community to detect potentially problematic images early on, peer reviewers need to equip themselves with skills to identify red flags during the review stage itself. Reviewing research that claims major impact on science and society may sometimes overwhelm peer reviewers. Thus, it is especially crucial to perform a robust scrutiny of image data in manuscripts reporting novel path-breaking results, because of the potential harm caused by erroneous claims. This is challenging because reviewers are not always formally trained in such inquiry.
As research frauds have made their way up to journals with the highest impact factors, discipline-specific expertise is not sufficient to detect these. It is now also important to develop a keen eye for fake images and the ability to judge if the image data provided are reliable enough to support research claims.
Steps to develop skills required to identify problematic images
1. Familiarize yourself with journal guidelines
Many journals provide guidelines on what they consider inappropriate beautification or cutting and stitching of images. Before you review a manuscript, familiarize yourself with the requirements and norms of the journal you are reviewing for.
2. Learn about the most common types of falsified images
Elisabeth Bik, an image-integrity expert, describes three major categories of problematic images8:
- Duplication or use of the same image in a single figure panel or across figure panels to report different experimental conditions
- Duplication along with repositioning of an image or a part of it
- Duplication with alteration involving “stamping” (a specific area duplicated multiple times) or “patching” (part of an image covered by an area with a different background)
3. Educate yourself about image forensics tools
Software that helps in manipulating images can also be helpful in detecting inappropriate manipulations. Such tools include Image J, frequently used to detect errors in gel pictures; Photoshop droplets, which can catch light/contrast adjustments; and Adobe bridge, which is used for reviewing large image panels. Forensically—a set of free tools for digital image forensics9—has helped Elisabeth Bik spot several problematic images10. You may also visit the websites of the HEADT Centre11 and the Office of Research Integrity (ORI)12 to check how software is used to spot tampered images. You may not be directly using such tools yet—and some are not free—but familiarizing yourself with how they are used will sharpen your ability to detect doctored images.
4. Attend workshops and webinars on research integrity
Several organizations, including ORI and the HEADT Centre, organize workshops and webinars to generate awareness about image manipulation and ways to tackle it. Watch out for such opportunities.
5. Follow research-integrity professionals and science journalists covering these topics on social media
Read up on the work being undertaken by several research-integrity professionals. Many of them discuss their observations, results, and learnings on their blogs or social media accounts. Science communicators and journalists often share any articles they write on research fraud and their interviews with data-integrity professionals on social media. Following a network of professionals via these channels can help you stay updated on evolving methods to detect image manipulation.
6. Visit online platforms where researchers identify problematic research Discussion forums like PubPeer13 and blogs like Retraction watch14 are online spaces that highlight and discuss recent cases of fraud or potentially problematic research, including image manipulation.
7. Connect the important dots
When reviewing a manuscript, focus on understanding connections between the research context, the methodology adopted or equipment used to generate image data, and the manner in which authors use the data to support a research claim. Careful attention paid to these connections can help you to spot any inconsistencies or red flags better.
8. The best critical lens are your eyes
A good way to start is to carefully study an important image (the one that ties most crucially with the research claim). You may look for
- Any edges that appear different
- Glows, extra-clean background, color variations that may suggest alteration/addition of objects in an image
- Consistency in size, angle, positioning of individual objects
- Similarity or overlap in textures and patterns of objects or background elements
9. Use your research experience to your advantage
- The authentic images you encounter in your own research experience can serve as benchmarks to compare any suspicious images with. Pay attention to aspects that seem different from what you typically see in true images. Use Google’s image search function to find pre-existing images if you suspect that the images are not original.
- Draw insights based on your memory. For example, if you regularly review manuscripts with Western blots, train yourself to remember if you have seen a similar images/image features in case you suspect a problem.
- Leverage your own image-editing experience. Photoshop tools can reveal odd pixelation in images suggestive of deliberate alterations. The picture reset tool in PowerPoint can help you find underlying original images.
Communicating concerns to the journal editor
As a peer reviewer, you have the ethical responsibility to protect research integrity by reporting any suspicions to journal editors. Here are a few pointers on how to go about this:
1. Treat raising a concern as a professional duty towards science. Reporting a concern about the work of fellow researchers may seem a highly unpleasant prospect. So, treating it as a professional obligation may help overcome any feelings of being overwhelmed or anxious.
2. Be objective. Your comments should focus on describing the suspected flaws in an honest, clear, and specific manner. Your aim should be to provide the journal editor enough context and detail to help them decide their next steps.
3. Be kind and cautious. Remember that you are raising a suspicion and not delivering a verdict. Thus, avoid making personal remarks about the authors or claiming intent to commit fraud. Word your report in a neutral and appropriate manner and leave room for a just inquiry.
Automated detection of image manipulation may soon serve as a more efficient way to prevent potentially fraudulent research from becoming part of the scholarly record. Until then, the contribution of peer reviewers in doing this will be crucial. Additionally, flagging of problematic images by reviewers can enrich the data bank of false images and eventually support efforts to automate their detection.
References
1. Wang, L., Zhou, L., Yang, W. & Yu, R. Deepfakes: A new threat to image fabrication in scientific publications? Patterns 3, 100509 (2022).
2. Noorden, R. V. Journals adopt AI to spot duplicated images in manuscripts. Nature https://www.nature.com/articles/d41586-021-03807-6 (2021).
3. Lemonick, M. D. The Rise and Fall of the Cloning King - TIME. TIME http://content.time.com/time/magazine/article/0,9171,1145236,00.html (2006).
4. Terry, M. Former Pfizer Cancer Scientist Gets All 5 Papers Retracted. BioSpace https://www.biospace.com/article/former-pfizer-cancer-scientist-gets-all-5-papers-retracted-/ (2017).
5. Schiermeier, Q. German research centre widens misconduct probe | Nature. Nature https://www.nature.com/articles/news.2010.671 (2010).
6. Rossner, M. & Yamada, K. M. What’s in a picture? The temptation of image manipulation. J. Cell Biol. 166, 11–15 (2004).
7. Guglielmi, G. Peer-reviewed homeopathy study sparks uproar in Italy. Nature https://www.nature.com/articles/d41586-018-06967-0 (2018).
8. Bik, E. M., Casadevall, A. & Fang, F. C. The Prevalence of Inappropriate Image Duplication in Biomedical Research Publications. mBio 7, e00809-16 (2016).
9. Forensically, free online photo forensics tools. https://29a.ch/photo-forensics/.
10. Turan, J. The science and art of detecting data manipulation and fraud: An interview with Elisabeth Bik. The Physiological Society (2020).
11. Thorsten S. Beck. HEADT Centre - How to Detect Image Manipulations? Part 3. The Headt Centre https://headt.eu/How-to-Detect-Image-Manipulations-Part-3/ (2017).
12. Forensic Tools | ORI - The Office of Research Integrity. https://ori.hhs.gov/forensic-tools.
13. PubPeer - Search publications and join the conversation. https://pubpeer.com/.
14. Retraction Watch. Retraction Watch https://retractionwatch.com/.
Additional reading
Beck, Thorsten S. Shaping Images. Scholarly Perspectives on Image Manipulation. (Humboldt-Universitat zu Berlin, 2016)
Note: Interested in learning more about how peer reviewers can support research integrity by identifying problematic images? Listen to the views of an expert panel, including Elisabeth Bik (independent science integrity volunteer) and Michael Streeter (Director of Research Integrity & Publication Ethics, Wiley).
Comments
You're looking to give wings to your academic career and publication journey. We like that!
Why don't we give you complete access! Create a free account and get unlimited access to all resources & a vibrant researcher community.
Subscribe to Conducting Research
