Why I review lots of papers
Peer review is often held up as a “gold standard” for assessing the quality and rigor of scientific publications. Topical experts are asked to scrutinize the work, assess its suitability for publication, and make recommendations for improvement. Two things are a little weird about the process: (1) the reviewers are almost always doing this work for free, and (2) they are often anonymous. I’ll start by explaining why I’m willing to put so much time into this task with very little direct reward, and I’ll finish briefly with some thoughts on anonymous peer review.
Why work for free?
Over the past two years, I’ve reviewed at least 23 papers and acted as handling editor for 28 additional papers. During a similar period (papers published 2018 or 2019), I’m included as an author on 12 papers (3 as first author, 4 as senior author). I’ve reviewed a lot more than I’ve submitted, and there is an opportunity cost to all this work. I could certainly be spending the same time writing proposals, working with my own students, or working up data from my lab.
I feel obligated to do my fair share: I’ve seen posts on social media that encourage scientists to review one paper for each one submitted. However, in my case, and in case of many of my peers, I feel that this “submit one, review one” approach is insufficient. Each manuscript I submit is reviewed by at least two, and most commonly three experts. So, to achieve balance, each paper should be balanced by about three reviews. A colleague correctly pointed out that most papers have more than one author so the burden can be shared. This is true, but many of my co-authors are students and postdocs. While Ph.D. students and postdocs are sometimes asked to review papers, the burden appropriately falls more heavily on more experienced faculty researchers. Also, it is my guess that native English speakers are more often asked to review manuscripts for journals written in English. As a native English speaker, I have a considerable advantage in publishing my work, and it is relatively easy for me to review the quality of the writing in manuscripts that are written in English. Evaluating and helping to clarify the quality of the writing only part of a reviewer’s job, but it is an important part.
I’ve had to recruit reviewers: Currently serving on the board for Biological Bulletin (and previously for Integrative and Comparative Biology and Journal of Steroid Biochemistry and Molecular Biology), I’ve had to recruit reviewers. Sometimes this is hard to do…people are busy. As an editor, I appreciate all the thoughtful reviewers out there.
I’ve benefitted from peer review: The reviewers always find something I’ve missed. I particularly appreciated a pair of reviews I received for a paper I wrote last year about Molecular Physiology of Copepods. One reviewer was deeply knowledgeable about aspects of the literature I had glossed over. The other reviewer was able to step back and make insightful high-level suggestions about how the whole paper could be re-organized and re-framed for greater impact. These comments greatly improved the manuscript. It seems only fair to try to help others in the same way.
Sometimes I learn interesting things: Reviewing is a chance to get an early look at emerging research and potentially to learn where the field is going.
It can be a great training opportunity: Reviewing on my own can be kind of tedious. Every once in a while, I get asked to review a paper that I think would be interesting to a student or postdoc in my lab. In those cases, I ask the editor for permission prepare the review collaboratively with my lab member. Working together on the review creates a richer perspective in the review itself, and provides an opportunity for mentorship in this important aspect of academic research.
Why remain unnamed?
Many arguments have been put forward against traditional anonymous peer-review, in favor of a double-blind system, or more frequently open review. Some journals offer open review as an option, some require open review, and some even require open review with the reviewers finally specifically “endorsing” the final paper. Among the many arguments for open review are forcing the reviewer to take more accountability, promoting civility, and increasing transparency. Some of these thoughts are nicely distilled in a 2018 perspective within EOS https://eos.org/opinions/avoiding-the-guise-of-an-anonymous-review.
A key conclusion of this piece is, “To promote an open scientific dialogue, protect trust in the scientific community, and mitigate the burden on reviewers and editors, beginning reviewers should be trained through reviewer guidance workshops to have the courage, honesty, and dignity to sign their reviews without being fearful of reprisal.” I agree with aspects of this statement but disagree with or am confused by others. I see no way that signing a review mitigates my burden as a reviewer. Further, some journals (recently Frontiers in Marine Science) require reviewers to engage in a “collaborative dialog” with authors. This could be interesting, but certainly adds to reviewer burden. I struggle with the concept of signing reviews “without being fearful of reprisal.” Reprisal is a genuine concern, particularly for junior scientists. And even if the risk of reprisal is small, it makes the (uncompensated) duty of reviewing even less appealing. Beyond the potential for direct reprisal from the authors, I see relatively little value to a reviewer in having his or her name directly listed on the publication. I would put many of the papers I review in the category of “reasonably well-executed but not particularly interesting.” If they meet the journal’s standards for publication, I recommend publication, but I don’t feel inclined to “endorse” the work and resent being required to do so.
I consider myself a constructive reviewer. I’m probably easily identifiable…I tend to be verbose and detail-oriented. I sometimes do suggest additional papers to cite, but they are almost never my own. I sometimes comment on my own papers if I feel that they are mis-cited. When I have participated in open review processes, it has universally caused increased anxiety. Even if I am overwhelmingly supportive of the paper, I wonder if the authors will take exception to some small comment. I spend extra time finessing the wording. If I take a hard look at myself, I think I become “softer” (less rigorous) than I should. I try not to let the prospect of open review affect me, but I think I pull punches. And this is 4-years post-tenure!
In short, I’m not sure that the current predominant system of anonymous peer review is the best, but I see strong disincentives to a simple open review process.
Parting thoughts
I’m a diligent reviewer because it’s an important and necessary responsibility. I’m learning that as I advance in my career, I’m asked to review more and more things…papers from journals, proposals, work by students and postdocs, institutional policies. Reviewing takes time and mental energy, and there is an opportunity cost to these efforts. Journals and journal editors would be well-served to consider how to incentivize the process and make it as painless (pleasant?) as possible for the reviewer.