Saturday, May 02, 2015

Anonymity in peer review

As anyone who follows science-related social media knows by now, a PLOS journal recently rejected a paper by Dr. Fiona Ingleby, based partly on a reviewer who stated that the paper needed a male co-author.  This is appalling, and the response so far is that PLOS has removed the handling editor and removed the (anonymous) reviewer from there reviewer database.

Of course, since we have no idea who this reviewer is, we can only presume that he (Or she, at least theoretically) is still out there in the community, able to inject this sort of bias into other academic peer review systems at other journals, grants, etc.

 This has renewed discussion about anonymity and the appropriateness of either signed or double-blind review. The problem with double blind is that, especially in some fields (like analytical geology), it is fairly easy for the reviewer to guess the identity of the authors. The critique of signed reviews is that they allow retribution and might scare junior researchers into not challenging senior colleagues whom they rely on for recommendations, grant funding, etc.

Being scientists, we should test these hypotheses instead of arguing about them.  And luckily, the EGU open-access, open review journals should allow this opportunity.

For people not familiar with this publishing model, the open review journals (for example, Atmospheric Chemistry and Physics) post the original submitted manuscript, the reviewer comments, and the author comments online in the associated discussion journal (Atmospheric Chemistry and PhysicsDiscussions).  The final, revised manuscript is then published in the main journal.

The reviews in the discussion paper are a mix of anonymous and signed. So assuming that scholars of science publishing can come up with a criteria for what constitutes a soft review, it should be possible to apply that criteria to the database of published comments. So there should be data on the effect of (optional) signing vs anonymity here to be mined by interested parties. 


Isotopic said...

I'm not a big fan of the open review journals, they tend to be a bit of a mess. I'm also not a fan of anonymous reviews - I'd rather see it as a rare phenomenon, available if needed, but more of a nuclear option. I've always signed everything I'm allowed to (I even managed to get my name onto a nominally double-blind review once - I stuck my name in the text of the review and the editor double-checked with me that I wanted it to stay). I find that it helps me write more thorough, well justified reviews if I know my name is being attached to it. If someone gets their nose out of joint because I dumped on their crappy paper, that's too bad.

Chris Phoenix said...

I don't follow social media, so this is the first I'd heard. Disgusting!

If you want a poster-child canonical example of ugly destructive sexism that should be killed with fire, this reviewer's comments are it. Firing the editor who let this happen is completely appropriate. I wish more could be done to keep the reviewer from spreading this kind of harm.

To reject a paper (written by women) which points out that women publish less than men... on the grounds that women might publish less because women (allegedly) aren't as physically fit... Look in the mirror, Mr. Anonymous Reviewer. You are the problem!