An editor-in-chief's guidance on the benefits and features of good-quality peer review podcast

Transcript

An editor-in-chief's guidance on the benefits and features of good-quality peer review

Daniel Ridge (DR): Welcome to the Emerald Publishing podcast three part mini series on the peer review process. My name is Daniel Ridge, and I'm a commissioning editor at Emerald Publishing. In this episode, we're looking at the peer review process through the perspective of the editor-in-chief. To discuss this I'm joined by Dr Ann-Marie Kennedy, who's been editor-in-chief of the Journal of Social Marketing since 2023. She's Associate Professor of Management, Marketing and Tourism at the University of Canterbury, New Zealand, teaching and supervising master's and doctoral students. She has authored and reviewed numerous papers in her field across several high-profile publishers, including Emerald, Sage and Taylor and Francis. So thank you. First of all, thank you for joining me today. What I wanted to begin with was asking you to, maybe as a warm-up question, to walk us through the process of peer review from your perspective. 

Ann-Marie Kennedy (AMK): So for the journal, I'm the editor-in-chief, so Journal of Social Marketing, the first point of call as we read the articles as they come through is to see whether we can, if we have appropriate reviewers for those articles. Because our goal, I suppose, is that we find the best people to review each paper so that the authors actually get good feedback. So that's our first thing that we do think about, OK, is there, do we have appropriate reviewers for this article? And then secondly, we set about looking for those reviewers and checking if they're available, and also checking how much they've reviewed in the past for us, so that we're not overloading each reviewer, and then sending out that invite to them.

DR: Well, how often do you have the same reviewer review for your journal? Is it a few times a year? Or what do you think is too much?

AMK: So we often, when we have very good reviewers, we would ask them to review maybe four or five times a year. But if a reviewer is getting up to that many times, I would be looking to ask them to join our editorial review board so their reviewing is really recognised publicly. But in saying that I have seen people asked to review seven times a year, I think that's getting a bit excessive, because even being an academic myself, I don't know how I would deal with that many requests each year, so I think that's getting a bit excessive. But all in all, I would say most reviewers would get one or two from us a year.

DR: So what makes a good review itself?

AMK: So the bare basic of that is that the reviewer knew about the topic and that they read the article thoroughly. Because, sorry, I'm going to go kind of the opposite of your question and then circle back to what's good, but it's kind of sometimes easier to say what is bad. But so a poor review is where someone hasn't actually engaged with the article, and then they give the authors recommendations that are not appropriate. And so from my perspective as an editor, what I want is that that author to get really good feedback. So even if they didn't do the revisions and they withdrew the paper, it would still improve their writing, and it would still improve the article. So if you have someone who hasn't engaged and they give very broad kind of recommendations like, add more to your literature review, but then they don't provide any details on that, that's really hard for an author to actually follow through with. So then I guess on the opposite side of that a good review would be one where the people the reviewer gives specifics. So they say, you know, you've missed some literature in your literature review. Here are some examples. Here's some citations of some of the literature that you've missed. Or this is the topic that you should look for in, you know, Google Scholar or whatever you're using as your search engine. And then when they're having recommendations and like methods, they don't just say, you know, a vague ‘your method is not rigorous’. They would say ‘please provide more information on XYZ’. So the more specific that you can be in what the author needs to do, the more likely they are to be able to actually do those revisions. And that's the key thing, right? We want to help, as I say, my key aim is to help authors improve and be able to improve their work in an encouraging way as well. So I guess that's the last thing that I would say, that a good review is one that is not rude or mean or derogatory, because you know that that author is a person is as you are as a reviewer. So we want to treat them with respect. So always having, it's not necessarily having to be positive the whole way through, but it's more about respecting that they have actually put a lot of work into this, so to acknowledge that and then give them things to help them improve.

DR: So do the reviewers fill out a form where they ask them questions? Or is it that you request that the reviewer track comments on the document itself?

AMK: For the Journal of Social Marketing, there is a form, and so there are specific areas that we would ask them to give their recommendation on or their view on. We actually ask them in a question format. So have they XYZ? Or has this article shown you know, engagement with the literature, for instance, is one of the kind of the gist of one of them, and then they have an open section where they can just write whatever they want. So what people often do is, with those section, the questions, they might say a short yes or no, and then in the open box section, they would then give their really detailed review, where it might be broken down into, you know, introduction, good introduction, XYZ is missing literature review, you know, sufficient literature review details. And then they go through every section, and they might hit their comments in that open box.

DR: How would you expect an author to respond to a reviewer's comments?

AMK: Oh, this is a great one, because this is something that I try to teach my PhD students as well. When you're responding, the thing that I think you have to remember as an author is that this reviewer is doing this almost out of the kindness of their heart, right? They are giving to the discipline, giving of their time and their knowledge to the discipline. So when you're responding the same as the reviewer should do to you, you should treat them with respect. What we want to see is clear responses to each of the reviewer’s recommendations. So for instance, you might use a table, and so you would copy and paste the reviewer comments into the first column of the table, and then in the second column of the table, you would say how you've responded to each of those recommendations. And in the third column, you might have a copy and paste a quote from your revised manuscript to show that you have done it, and that's the easiest way for the reviewer to then just look down and go, tick, tick, tick. Yes, you've responded to all of my recommendations.

DR: Now, what if an author wants to become a reviewer? Is it possible to contact an editor of a journal to volunteer as a peer reviewer?

AMK: Absolutely. We are always looking for reviewers. So that's exactly how you would do it. You would just email one of the editors. It could be one of the associate editors, regional editors, editor-in-chief, and then we can add you to the list. I think the really good thing to do, though, is you can go on and register for the journal anyway. So you create your own profile on the journal, and when you create your own profile, there's a section in there that asks you your research interests and what your area of expertise are. And when we look for reviewers, we search through the system for people who have the area of expertise that the article is in. So, you know, actually, the best, best way of becoming a reviewer is by having an up-to-date profile in our journal so that we can find you, basically, so we can put in a search. You know, gambling, for instance, gambling harm, and we find there you are, there's your profile, and you've got that all up to date. 

DR: So who should become a reviewer? Who do you think? Do you think that there's people who are more qualified to review than others? You know, somebody who's early career, later career, mid career? What does that have to do with reviewing?

AMK: So I think anyone who is an academic can become a reviewer, and possibly should become a reviewer, because, as I said before, in my opinion, it's giving back to the community. And the reason that I say it in that way is because a reviewer is shaping future academics, so whether they're a current one, but they're shaping what it looks like to have a good journal article. They're shaping what knowledge is chosen to be included in articles, what is ignored. They're shaping standards and rigour in your discipline. So I think anyone who is an academic should do reviewing so that they can contribute to the whole ecosystem. And in terms of early career researchers, I think it's really important that early career researchers and even PhD students also become reviewers. From my own personal experience, that was the way that I learned how to write a good journal article. Because when you review an article, you get to see what standard other people have and what structures they're using, and different methods and knowledge categorisation, but also often, and our journal is one of the ones that does this, what you get when the editor sends the decision to the authors, they also send an email to you as a reviewer and the other reviewers, and they attach all the reviews to that email. Obviously, everything is anonymised, but what it means is, you as a reviewer, can read the other reviewers’ comments on the article. 

DR: Oh, wow. OK, I didn't know that. 

AMK: Yes. And so the amount of learning that I have done, and I still do through reading the other reviewers’ comments is, you know, astronomical, because you're finally, you're seeing, oh, OK, I was a little bit harsh on that review, or I was way too easy. Or you read the review and think, oh, I didn't even catch that. So I think that is one of the biggest rewards for being a peer reviewer is if you get to read the other reviewers’ comments.

DR: Yeah, that's great. Well, working with your students. What advice do you have for them?

AMK: I think the thing is to be constructive in your comments. So what I always see in early career researchers, nearly always, is there's they are either in two camps. One, they just say everything is great, and they accept everything on the first round, and they don't really engage with the article, because they think they don't have anything to contribute. So they don't say anything. And the other end is that they're super harsh and saying, reject, reject, reject. This is no good. There's all these problems with it. So my general feedback, or like advice to them, is to consider whether what you're saying is doable for the authors. It's almost that kind of old, you know, adage of, how would you feel if you were an author and you received that review, would you cry? Would you be happy about it? Is it helpful for that author? So thinking about how would you feel if you got that review is a good starting point.

DR: Thank you for listening to today's episode. For more information about the peer review process and to listen to other episodes in our peer review miniseries, please visit our website by searching Emerald Publishing podcast series, peer review. I'd like to thank my guests for joining me today and the studio This Is Distorted.