Anonymity in peer review

As anyone who follows science-related social media knows by now, a PLOS journal recently rejected a paper by Dr. Fiona Ingleby, based partly on a reviewer who stated that the paper needed a male co-author.  This is appalling, and the response so far is that PLOS has removed the handling editor and removed the (anonymous) reviewer from there reviewer database.

Of course, since we have no idea who this reviewer is, we can only presume that he (Or she, at least theoretically) is still out there in the community, able to inject this sort of bias into other academic peer review systems at other journals, grants, etc.

 This has renewed discussion about anonymity and the appropriateness of either signed or double-blind review. The problem with double blind is that, especially in some fields (like analytical geology), it is fairly easy for the reviewer to guess the identity of the authors. The critique of signed reviews is that they allow retribution and might scare junior researchers into not challenging senior colleagues whom they rely on for recommendations, grant funding, etc.

Being scientists, we should test these hypotheses instead of arguing about them.  And luckily, the EGU open-access, open review journals should allow this opportunity.

For people not familiar with this publishing model, the open review journals (for example, Atmospheric Chemistry and Physics) post the original submitted manuscript, the reviewer comments, and the author comments online in the associated discussion journal (Atmospheric Chemistry and PhysicsDiscussions).  The final, revised manuscript is then published in the main journal.

The reviews in the discussion paper are a mix of anonymous and signed. So assuming that scholars of science publishing can come up with a criteria for what constitutes a soft review, it should be possible to apply that criteria to the database of published comments. So there should be data on the effect of (optional) signing vs anonymity here to be mined by interested parties.