Peer Review, and Lying Injured on the Sidewalk

Yesterday I spent over 3 hours reviewing a paper for a journal, capping off a jam-packed day of semi-improvised discussions with high school students, grant proposal writing, a very tiny bit of lab work, and more. A friend of mine, coincidentally, also spent several hours yesterday reviewing a paper. This morning, I read a blog post from Andrew Gelman wondering about why we do it, which I wonder about too, and contemplating the alternative of not reviewing papers before publication but rather relying on post-publication commentary. There have been a lot of discussions proposing something like this, scrapping prior peer review and its flaws and burdens, with a lot of arguments for and against. I realized this morning, however, a very strong objection to post publication commentary that I hadn’t been able to put into words before: post-publication review will fail, for the same reason that one doesn’t want to be lying injured on a busy city sidewalk. What do I mean?

Those pushing for post-publication commentary note that with such a setup, anyone who cares about an article can comment on it. A counter to this is that in the places presently set up where such comments are possible, there’s a remarkable lack of comments. Almost no one bothers to critique published papers. The reply to this is that this absence of commentary reveals that no one actually cares about the papers or their content. While it is true that we’d be better off if fewer papers were published, this conclusion is wrong. Why?

Bystander effects

It often happens that a crime, injury, or accident occurs in full view of dozens of people, yet no one takes the initiative to help. Perhaps the most famous (though likely exaggerated) example is the murder of Kitty Genovese in 1964, at which

For more than half an hour thirty-eight respectable, law-abiding citizens in Queens watched [from their apartment windows] a killer stalk and stab a woman… Not one person telephoned the police during the assault. (New York Times)

There are many such examples, including recent ones such as the death of Wang Yue in China in 2011, and plenty that are less dramatic. The well-known psychological phenomenon has a name, the Bystander Effect.

One could perhaps conclude from these cases that people are simply cold and awful. However, it is also the case that people, if individually called out, will very often willingly help others. If you’re lying bleeding on the sidewalk, you’re much, much more likely to get help by saying “You, sir, in the blue jacket, I need help. Call an ambulance,” than by expecting someone in the crowd to step up. (This particular phrasing is from Robert Cialdini’s Influence, which is a wonderful book, and is where I first learned about the bystander effect.)

The diffuse crowd of post-publication peer review

The connection to peer review is perhaps obvious, but I’ll spell it out anyway.

As mentioned, I spent a few hours reviewing a paper yesterday. I don’t know the editor (in fact, I didn’t bother reading who the editor is), though I’m familiar with the journal. I am, essentially, a stranger. However, as I do roughly once a month, I spent several hours thinking carefully about an article, carefully checking (and critiquing) its arguments, and writing hopefully helpful statements that would improve it, all for no personal benefit other than a vague feeling that it helps the field. The journal editor emailed me (and other potential reviewers) asking if we’d perform the review. I sometimes say yes to these requests, and sometimes I say no, but I always consider them.

In contrast, if the paper were just “out there” for commentary, I’m sure I would not have bothered to comment on it.

Am I a good person as a reviewer and a bad person as a non-commenter? No, just as the man in the blue jacket is fundamentally the same whether called out or not. Being called out, or being specifically asked to review a paper, triggers those circuits in our mind that generate a response.

Post-publication review, if that’s all there is, will languish not because no one cares about scientific articles, but because it turns us all into bystanders.

Today’s illustration

A drawing of membrane proteins at two contacting cells, which I drew for something else I’m working on.

— Raghuveer Parthasarathy. July 25, 2018