Thursday, April 02, 2009

Why do we publish papers?

I recently had a short technical note rejected by JAAS, wrote a bitchy self-indulgent blog post about it. Then through the miracle of delayed posting came back and revisited it before it went live. So, in an attempt to create something productive (and marginally less self-indulgent) out of the experience, I’d like to look into the final comment of the first reviewer:

2g. References: I could locate but not open Geostandards Newsletter and Geostandards and Geoanalytical Research journals. I could not find ICP-MS Journal 2000 (refs 1 and 1-10). Ref. 11 is not complete. I had also no access to refs. 4-7, 12, 13. Ref. 14 is missing.

Geostandards newsletter, which became GAGR in 2001 or so, was the publication referenced in numbers 2, 3 (mostly), and 9. Number 8 referenced an article in JAAS, the journal for which this person was reviewing. How many does that leave?

In reality, of course, it doesn’t matter if anyone ever reads papers. Papers are just a way ot scorekeeping for promotion reviews of professional academics. Back in the 19th and 20th centuries, however, journals were actually used for the dissemination of scientific information. I’m not old enough to know what that was like, but it does make me wonder:

If non-academic scientists get all old-fashioned and traditional, and feel the urge to give their colleagues tips, what is the point of trying to publish papers if even the people who review the papers can’t read any of the work on which the research is based?

The reference list is attached. Observant readers may note that my likelihood to be taken seriously as a scientist might improve if I could successfully demonstrate the ability to count to three.

1 C. Tye, K. Sakata, ICP-MS Journal 2000, 8, 7.
2 S. M. Eggins 2003 Geostandards and Geoanalytical Research 2003, 27, 147–162
2 K Sakata, N. Yamada, Naoki Sugiyama Spectrochimica Acta Part B, 2001, 56, 1249-1261.
3 K. Govindaraju. Geostandards Newsletter 1994, 18, 1-158; S.M. Eggins, J. D. Woodhead, L. P. J. Kinsley, G. E. Mortimer, P. Sylvester, M. T. McCulloch, J. M. Hergt, M. R. Handler. Chemical Geology 1997, 134, 311-326; M. D. Norman, W. L. Griffin, N. J. Pearson, M. O. Garcia, S. Y. O'Reilly, Journal of Analytical Atomic Spectrometry, 1998, 13, 477-482; R. W. Hinton, Geostandards Newsletter: The Journal of Geostandards and Geoanalysis 1999, 23, 197-207; S. Gao, X. Liu, H. Yuan, B. Hattendorf, D. Günther, L. Chen, S. Hu, Geostandards Newsletter: The Journal of Geostandards and Geoanalysis 2002, 26, 181-196.
4 S.-s. Sun W. F. McDonough. 1989 Chemical and isotopic systematics of oceanic basalts: implications for mantle composition and processes Geological Society, London, Special Publications; v. 42; p. 313-345
5 J. Longhi, American Journal of Science 1987, 287, 265-331.
6 D. L. Hamilton, D. M. B. Henderson, Mineralogical Magazine 1968, 36, 832-838.
7 S. Eggins, R. Grün, A. Pike, J. M. S. Shelley and L. Taylor, Quaternary Science Reviews 2003, 22, 1373–1382.
8 H. P . Longerich, S. E. Jackson and D. Gunther, Journal of Analytical Atomic Spectrometry, 1996, 11, 899-904
9 N. J. G. Pearce, W. T. Perkins, J. A. Westgate, M.P. Gorton, S.E. Jackson, C. R. Neal, S. P. Chenery, Geostandards Newsletter, 1997, 21, 115-144.
10 C. Tye, K. Sakata, ICP-MS Journal 2000, 8, 7.
11 I. Parsons, C. Magee, C. Allen, J. M. S. Shelley, M. Lee, Mutual replacement reactions in alkali feldspars II: Trace element partitioning and geothermometry. Contributions to Mineralogy and Petrology. In Press.
12 A. Kallio, T. Ireland; “Silicate melt inclusions in komatiites as potential indicators for crustal growth”. 2006 16th annual Goldschmidt conference.
13 J. D.Stopar, G. J. Taylor, and M. D. Norman 2007 Aqueous alteration in Naklite MIL 03346: LA-ICPMS and Raman spectrometry. 7th International Mars Conference.

* * UPDATE * *

Can you find the references listed above? If so, can you read them? There are polls to the right! -->

10 comments:

Kim said...

I haven't tried from work, but at work I've only got access to Geology, GSA Bulletin, and a few databases (which don't include many older articles). I force students to use interlibrary loan, and will continue to do so until most research is published in open access journals. I'm very worried that I'm training a generation of geologists who will only ever read articles available through Google Scholar.

BrianR said...

"I had also no access to refs. 4-7, 12, 13."

WTF! A reviewer complained about not having access?! That's bullsh@t. Sounds like one of these internet crank excuses. What a waste of a reviewer.

To any grad student reading this that may one day be asked to be a reviewer ... don't be an ass like this person ... do the work!

David Raikow said...

I have to disagree with your statement "it doesn't matter if anyone reads papers". We are trying to elucidate the true nature of reality. If no one reads your paper, the work functionally does not exist, and thus does not contribute to greater understanding. The classic example is Mendel, who’s work was essentially not read for decades. How far could we have advanced if he was read in his time instead of being re-discovered decades later?

That said, I too hate the reviewing process. I’ve had more than my share of low opinions of reviewers. I just got rejected without of review, so I have to guess at what they didn’t like.

Kim said...

I'm at work now, and I searched for some of the articles. I could find most of them on Georef, and I had access to a couple of them. (Mineralogical Magazine has scanned its archives and made them freely available! Cool! And ICP-MS Journal isn't in Georef, but I think that's a limitation of just using Georef, rather than trying some indexes that would include chemistry as well as geology.)

But that's a limitation of my institution, not of your paper. I can't imagine giving someone a bad review because of the limitations of my institution. (Maybe if I were at a major research university and I wanted to make sure that the references really existed... but I usually focus on the methods and results and interpretation, rather than double-checking all the references.)

Anonymous said...

One trick may be to add the doi (digital object identifier) number to the references where it exists. That way if a bone-idle reviewer receives an electronic version of the manuscript then the reference, or at least the website with potential access to the reference (depending on their institute's access rights), will come up with the click of the mouse button.
Also assuming the reviewer knows how to use a mouse of course.

Dr. Lemming said...

ICP-MS journal is the instrument manufacturer's newsletter- not peer reviewed, but a handy place to find technical tips and settings.

Copy and pasting that reference into normal google (not scholar) should do the trick.

Kim, what's the fee on interlibrary loan where you are? It might save them $$ by just paying the online access charge, depending on the journal.

Anonymous said...

I'm surprised they'd even bother to send you a reject notice. Talk about a reject.

C W Magee said...

I see that anonymity raises the level of debate in blogging as much as it does in the peer review system...

EliRabett said...

Once more the editors don't do their job. Reviews like this should be accompanied by a letter from the editor saying never mind on this point. If necessary they should solicit another review.

We need better editors

Anonymous said...

Sounds like you got a high school student as a reviewer. If I were the editor I'd tear into the reviewer - not normally done, but who wants to ask a useless reviewer to waste everyone else's time?

Dang, that reminds me. I still have 4 pages to go and then I need to review and write up my comments and also check a few facts in cited journals which I won't whine about not having access to.