Should reviewers phone a friend?
Role playing time, lemminglets. Suppose you are reviewing a paper. Also assume, that like most papers these days, that it has multiple authors, each of whom applies his expertise to the problem at had. And finally, assume that you are an expert in some, but not all of the fields used to solve the particular problem being reported in this paper.
What do you do if one of the key points in the paper that is not your area of expertise seems fishy. For example, if the paper is on your field area, what if some of the lab results seem fishy. Or if you are an analyst, what if the experimental setup seems odd.
Assuming that you are a successful researcher, you probably have long-time collaborators who are experts in these fields. So, what is the best way of accessing their expertise, given that some sort of confidence generally surrounds papers in review.
By ‘best’ I mean best for science, but if any of y’all want to interpret this as ‘best for me’ or ‘best for the editor’ or ‘best for the author’, that is fine too.
11 comments:
Yeah, that's a tough one ... I've consulted colleagues in situations like this, but for pretty general aspects of the scientific argument. When it comes to specific details of a paper, however, I don't share too much. You could make sure to note your thoughts (i.e., that it seems fishy) in comments to editor and hopefully another reviewer is an expert in those aspects and caught it. If not, then an editor might let it get through.
Good luck.
I know that in extreme cases, my advisor has asked the editor if she can consult with a colleague on certain aspects of the work.
I asked a close colleague for help when I reviewed a paper that had a technique I wasn't super familiar with. I concealed it somewhat by asking something like, "when people make X measurement, is it okay to use method B instead of the more typical method A?" That got us into a general discussion that I could use to inform my decision. Still, I bet she knew I was asking for a reason like a review (I might have even said as much, although I did not describe the paper in detail).
I'm curious to hear others' opinions since I have little experience on this. But I felt pretty comfortable that what I did was ethical.
So is protecting confidentiality more important than getting the science right? Also, does the anonymity (or lack thereof) of the review matter at all?
IME journals are quite happy for reviewers to ask a colleague for advice. GRL have a box in their web system for "did you consult with anyone" and I think many others explicitly mention it as a possibility. No problem so long as the consultee also respects the confidentiality.
On a similar subject, just a couple of days ago I was sent a whole ms to review, cold, which was on a wholly inappropriate subject (from one of these junk spam journals, nothing respectable). Needless to say I declined...
Cold, as in no prior contact from the handling editor? That's kinda odd.
I'm trying to imagine a wholly inappropriate spam manuscript, but all I'm getting is statistical tests on the efficacy of enlargement medication...
It is completely inappropriate to consult with anyone other than trainees in your own laboratory concerning paper review without discussing it with the editor first. This is a completely bright-line ethical rule.
I believe that a reviewer who does not have enough expertise where one aspect of a manuscript (or a grant proposal) or another is concerened, must consider consulting colleage experts. I remember serving on an NIH study section, reviewing grant applications and having problems with one of them, where the PI made several assumptions on which he based his prediction of the expected outcome. Though these assumptions seemed wrong to me, I did not think that my expertise in that particular area was up to par. I phoned an old colleague who worked in that very area, explained to him that I am in the middle of reviewing a grant proposal. I explained to him the assumptions made by the PI (not revealing personal data) and in return I received a detailed explanation plus a list of references to read on the topic. Consequently, I became much better equipped to evaluate the PI's assumptions and to determine that he was full of crap.
It is totally appropriate to get help in reviewing a paper... HOWEVER, it is also absolutely critical that you get permission from the editor first. Just write an email to the editor saying "There's an issue I'm concerned about that I would like to discuss with my colleague Dr. X who is an expert in this area." The editor will say "OK, but make sure Dr. X maintains confidentiality." and then you're good to go. I've never seen an editor turn down such a request.
PS. It is also important to tell the editor that you would like to consult/co-write the review with a trainee before the trainee sees the manuscript.
As CPP says, this is an ethical bright line. But it's such an easy one to deal with, there's really no reason not to just ask.
I phoned an old colleague who worked in that very area, explained to him that I am in the middle of reviewing a grant proposal. I explained to him the assumptions made by the PI (not revealing personal data) and in return I received a detailed explanation plus a list of references to read on the topic.
Shitlin, this is completely unethical. Nice moral compass, hypocritical sleazeball.
I don't see why some posters need to feel such self-righteous outrage. It's quite possible that policies vary by journal and/or by field.
I believe Nature is also comfortable with reviewers consulting other people, although I am not 100% sure about that (eg they may prefer to be asked first, I don't recall ever being in that situation).
If the reviewing instructions don't explicitly allow consulting with colleagues, my opinion is that you should ask the editor. Trying to ask a colleague about it in some vague way that you think doesn't break confidentiality seems risky to me - if you don't know enough to know whether what the authors did is right or wrong, can you be sure you're being vague enough?
I'd be pretty surprised if the editor didn't approve the consultation, but a quick email to ask seems appropriate and painless.
Post a Comment