Podcast: Why peer reviewers should develop feedback literacy: Interview with Sin Wang Chong

If you were asked to list the most important attributes one needs to have as a peer reviewer, what would they be? Very likely, you’ll talk about subject matter expertise, critical-thinking skills, and the ability to provide a sound, objective assessment on the quality of research presented in a manuscript. Which are all, no doubt, imperative.

But most of us rarely think of peer review as performing an important social function, as well as an intellectual one—of supporting fellow researchers in your field and building collegial relationships through peer review feedback. That’s where the concept of “feedback literacy” comes in.

In this interview series, I speak with Professor Sin Wang Chong, who has done extensive work exploring the value of feedback literacy in scholarly peer review. This year’s Peer Review Week focuses on the theme Peer Review and the Future of Publishing, and so, I ask Sin Wang how this particular social value of peer review can be strengthened and what direction he believes peer review should take in the future.

In this first segment, Sin Wang describes what feedback literacy is and why it’s important in peer review and scholarly publishing.

In the second segment, we discuss how researchers from all linguistic backgrounds can learn to provide constructive feedback (even though English is the primary language used for global scholarly communication) and the potential ways AI can help peer reviewers improve their feedback quality ethically.

In the third segment, Sin Wang talks about the importance of shaping the future of peer review based on the principles of inclusivity and empowerment, in particular, empowerment of early career researchers.

About Sin Wang Chong: Sin Wang is Director of Impact and Innovation at the International Education Institute, University of St Andrews, and Head of Evidence Synthesis at the National Institute of Teaching in England. Concurrently, he is a visiting and adjunct professor at a number of universities in Asia, England, and the United States. 

He is Chair of the Scottish Association for Teaching English as a Foreign Language (SATEFL) and serves on the Council of the British Educational Research Association and the Executive Committee of the British Association for Applied Linguistics.

Sin Wang’s research interests are in evidence synthesis, educational assessment, language education, and higher education. He is Associate Editor of two SSCI-indexed journals: Innovation in Language Learning and Teaching and Higher Education Research & Development. He is co-founder and co-director (with Shannon Mason) of Scholarly Peers, a platform to support doctoral students and early career researchers to navigate journal peer review.

[Audio transcript]

Mriganka

Hello. I’m Mriganka Awati, Senior Writer at Editage Insights. This interview is one of the activities that Editage Insights is conducting to celebrate Peer Review Week this year. And I’m speaking with Professor Sin Wang Chong, who is the director of Impact and Innovation at the International Educational Institute, University of St. Andrews, and he’s also the head of Evidence Synthesis at the National Institute of Teaching in England. His research interests are in evidence synthesis, educational assessment, language education, and higher education.

But he has also done very interesting academic work on peer review and has taken initiatives to help early career researchers navigate this process. Sin Wang, thank you for joining us this Peer Review Week.

Sin Wang

Thanks for inviting me.

Mriganka

So, this year’s theme for Peer Review Week is “Peer Review and the Future of Publishing.” And I’m really excited to speak with you because as the scholarly community thinks about how to strengthen this process, I think that the value of peer review as a form of communication between authors and reviewers—of giving and receiving feedback—is also important to consider.

So, here’s my first question. In an opinion piece in Learned Publishing, you wrote about how peer review can be improved by developing reviewers’ feedback literacy, and I found this quite an intriguing concept. So, what is feedback literacy?

Sin Wang

Yeah, thanks very much for the question. And I think before I begin answering the question, I think one thing I would say is that I think peer review, it’s an important topic to talk about.

I think it’s a topic that isn’t talked about enough, especially, you know, what happens behind the scene, in particular, the role of feedback in the whole peer review process, and feedback plays such an important role because it’s the facilitator and the mechanism through which manuscripts are improved and eventually published. So, I talked about this notion of feedback literacy, as you said in my piece, published in Learned Publishing. The concept of feedback literacy stems from the literature on assessment literacy and academic literacies, and it’s mostly discussed in an educational context.

And when we talk about feedback literacy in an educational context, we talk about student feedback literacy and teacher feedback literacy. The term first appeared in 2012 in Sutton’s paper, where he defined student feedback literacy as a set of generic practices, skills, and attributes, which is a series of situated learning practices. And then in 2018, David Carless and David Baud, in their very seminal paper, described three interrelated components of student feedback literacy, which I referred to in the Learned Publishing paper, and they talked about the importance of appreciating feedback, meaning how we understand the values and our roles in the feedback process, how feedback helps us make informed judgments, and also how we manage emotions.

So, three attributes here: Appreciating feedback, making judgments, and managing emotions. And in my paper in 2021, I proposed an ecological model of student feedback literacy that not only comprises the kind of three attributes that I talked about, but also looking at the contextual and the individual learner dimensions. And I argue in this model that feedback literacy is an outcome based on the interactions between contexts where learners are situated in and the individual differences of learners.

I also would like to talk a little bit about teacher feedback literacy before I move into feedback literacy for academics, which is what we want to talk about today. So, in Carless and Winstone’s paper in 2020, they defined teacher feedback literacy as three attributes of teachers. Firstly, teachers’ capacity to design feedback, uptake opportunities, how teachers address the relationship aspect of feedback, how teachers reconcile personal, disciplinary, and institutional expectations on feedback practices.

So as you can see, so far, the literature on feedback literacy focuses on education, focuses on learning and teaching in higher education. The first paper that talked about feedback literacy in an academic context for academics, it’s a paper by Gravett and colleagues published in 2020, where they talked about feedback literacy for academics specifically, and, in particular, in relation to journal peer review as authors and peer reviewers.

So in my paper, I picked up on that, you know, direction and argue that feedback literacy for journal peer review is essentially about two things. First, how peer reviewers can provide professional, constructive, and actionable feedback. And secondly, how authors can make the most out of the feedback they receive. So you see feedback literacy is not just for reviewers but also for authors.

Mriganka

Now let’s say I were to argue that the objective of peer review is primarily to evaluate the scientific quality of a manuscript, which is a task that already requires multiple skills that reviewers need to build and is quite effort-intensive and time-intensive. So, why do you believe that developing this specific competency—feedback literacy—why is it important and how can it help journal editors, reviewers, and authors and the research enterprise overall?

Sin Wang

Let me just say this. I completely agree that one of the objectives and possibly the primary objective of journal peer review is to evaluate scientific contributions and methodological rigor of a manuscript. However, I think it’s important that we don’t forget about the formative nature of journal peer review. What I mean by formative is how peer reviewers’ feedback can be used to help authors to develop their manuscripts further.

So, essentially we’re not just looking at the end product of a publishable, improved manuscript, but the process through which the manuscripts are improved. And I also concur that peer reviewing can be a complex process and one that requires multiple skills. And I would argue, in particular, when it comes to giving feedback, there is a lot of focus on the content of feedback.

For example, editors will look at researchers in the substantive and methodological areas of the manuscript to provide expert feedback. I think while these experts are definitely skilled in their own area, they may not be trained to provide feedback in a way that would facilitate the uptake of the recipient. That’s where feedback literacy is so valuable, because feedback literacy is not about what is included in the feedback, but how to deliver the content of the feedback in a way that facilitates uptake.

So, it is the same as the understanding that, for example, someone who is very good at something does not make them a good teacher of it. Teaching requires another set of skills, so does giving feedback for journal peer review. So, so returning to your question and I think it’s precisely because peer review is effort- and time-sensitive that makes developing reviewers’ and authors’ feedback literacy so important so that the precious time of academics does not go to waste.

So how can feedback literacy help journal editors, authors, peer reviewers, and research quality in general? To journal editors, I think with feedback-literate reviewers and editors, it ensures the publication of cutting-edge and well-developed and well-conducted research based on a very rigorous and professional peer review process, underpinned by high-quality feedback.

For authors, I think with feedback literacy, it ensures that feedback received is specific. Feedback is easy to understand. Feedback is helpful, manageable, and actionable.

For peer reviewers, it develops an awareness about what feedback is all about, who the feedback is for, and how to communicate complex information effectively to other researchers, and also, very importantly, how to separate professional judgments from personal preferences.

Finally, for research, I think it helps develop a sense of community, a sense that research is a collaborative effort, that we work together or researchers who have never met work together to improve the quality of research in publications in general.