Wednesday, January 24, 2007

The speed of science: relativistic media vs. classical research

As mentioned previously, I will be writing a comment on a paper which failed to impress me. My general complaints are that I think it misrepresents the literature, fails to describe its samples and analyses well enough to allow an objective interpretation, and overstates the quality of the data. But outside the ivory tower, life goes on.

Despite any previous metaphors used in this blog to describe the comment and reply process, it is one of the more standard, classical ways of discussing scientific findings. When done well, it is measured, fair, and provides greater illumination of how different research groups approach problems and weigh various evidence for and against competing hypotheses.

At the other end of the spectrum of scientific discourse is sound-byte and press release science. As it happens, the paper on which we wish to comment is accompanied by two press releases, by the university and by the NSF.

I think it may be interesting to compare the progress of buzz and reportage to the rate of constructing a scientifically sound counter-argument. So here’s a timeline:

Dec 20
The Astrophysical Journal Letters publishes the Garai et al. paper on carbonado diamond, which I think is a poor research publication.

Jan (date unknown, but probably the 2nd)
American Mineralogist publishes a Kagi et al. paper on carbonado, which I think is a contender for the best paper on the subject in the last 5 years.

Jan 8
Media:
NSF press release issued about Garai et al. paper
Articles on Garai et al. ‘06:
http://www.livescience.com/forcesofnature/070108_spacey_diamonds.html
Articles on Kagi et al. ‘07: (none)

Jan 9
Articles on Garai et al. ‘06:
http://www.diamonds.net/news/NewsItem.aspx?ArticleID=16479
http://www.infozine.com/news/stories/op/storiesView/sid/20089/
http://www.webwire.com/ViewPressRel.asp?aId=25971
http://www.shortnews.com/start.cfm?id=59413&rubrik1=Science&rubrik2=Research&rubrik3=All
http://www.physorg.com/news87577799.html
http://www.eurekalert.org/pub_releases/2007-01/nsf-dfo010907.php
http://www.foxnews.com/story/0,2933,242575,00.html
http://www.spaceref.com/news/viewpr.html?pid=21626
http://www.astrobiology.com/news/viewpr.html?pid=21626
Articles on Kagi et al. ‘07: (none)


Jan 10:
Science:
My postdoc adviser emails me the paper.

Articles on Garai et al. ‘06:
http://www.diamondintelligence.com/magazine/magazine.asp?id=4654
http://www.dailyindia.com/show/101783.php/
http://www.huliq.com/5491/origin-of-earths-mysterious-black-diamonds
http://www.sootoday.com/content/news/full_story.asp?StoryNumber=22052
http://www.swnebr.net/newspaper/cgi-bin/articles/articlearchiver.pl?159523
http://presszoom.com/story_122497.html
http://reports.discoverychannel.ca/
http://www.cosmosmagazine.com/node/960
Articles on Kagi et al. ‘07: (none)


Jan 11:
Articles on Garai et al. ‘06:
http://www.allamericanpatriots.com/m-news+article+storyid-17569.html
http://blogs.zdnet.com/emergingtech/?p=459
Articles on Kagi et al. ‘07: (none)

Jan 12:
Articles on Garai et al. ‘06:
http://www.innovations-report.de/html/berichte/geowissenschaften/bericht-76806.html
http://www.photonics.com/content/news/2007/January/12/85986.aspx
http://www.ccnmag.com/news.php?id=4724
Articles on Kagi et al. ‘07: (none)

Jan 13:
Articles on Garai et al. ‘06:
http://www.spaceflightnow.com/news/n0701/12diamonds/
Articles on Kagi et al. ‘07: (none)

Jan 14:
Articles on Garai et al. ‘06:
http://www.spacedaily.com/reports/
http://space.newscientist.com/
Articles on Kagi et al. ‘07: (none)

Jan 15:
Science:
I get back from summer holiday, check my mail, read paper. Scratch head, read again, explode. Scrape self off ceiling, email postdoc advisors and PhD advisor to see if they are interested in co-authoring a comment (My PhD was on carbonado diamond). I start digging back through papers I haven’t looked at for 6 years.
Articles on Garai et al. ‘06:
http://www.theregister.co.uk/2007/01/15/black_dimaonds/
Articles on Kagi et al. ‘07: (none)

Jan 16-19:
Science:
Post doc advisors get on board, I chat with one of them, we map out a general plan, start looking for the software/ archived files we need. I learn that Statview hasn’t been sold since 2002, start thinking up workarounds.
Articles on Garai et al. ‘06:
http://dsc.discovery.com/news/2007/01/18/blackdiamond_spa.html?category=space&guid=20070118103030&dcitc=w19-502-ak-0000
http://talkback.lancasteronline.com/index.php?showtopic=40034
Articles on Kagi et al. ‘07: (none)

Jan 20-21:
Science:
Go through paper (over weekend) with attention to detail, start categorizing things I don’t like by relevance, type of issue, dubiousness of claim, etc.
Articles on Garai et al. ‘06:
http://www.cleveland.com/news/plaindealer/index.ssf
http://abc.net.au/science/news/stories/2007/1831109.htm
Articles on Kagi et al. ‘07: (none)

Jan 22-present
Science:
Finally get some obscure articles from Geoscience Australia (not in our library) to confirm selective referencing, crash IR scope computer trying to remember how the OS/2 operating system works, get PhD supervisor interested, get swamped with work for visiting scientists.
Articles on Garai et al. ‘06:
http://news.nationalgeographic.com/news/2007/01/070122-black-diamonds.html
http://www.inform.kz/showarticle.php?lang=eng&id=147985
Articles on Kagi et al. ‘07: (none)

Draw your own conclusions.

2 comments:

Chris R said...

According to Wikipedia, science is in fact moving at the speed of light, and 'black diamonds from spaaaace!' has already become the 'currently accepted hypothesis'. The perils of science by press release writ large.

Out of interest, is your problem with these guys' methods, their interpretations, or both?

C W Magee said...

Those, as well as the quality of the spectra.

Here's the easiest problem to explain, though:

The first two sentences of the abstract are:

"The first complete infrared FTIR absorption spectra of carbonado diamond confirm the interstellar origin of this most enigmatic diamond. All previous attempts failed to measure the absorption of carbonado diamond in the most important IR range of 1000-1300 cm-1 (10.00-7.69 mum) because of silica inclusions."

As it turns out, an Open University-UCL team published an FTIR result almost 10 years ago, as part of an isotopic and IR study. The paper is Shelkov et al. 1997.

Shelkov et al. '97 is referenced by Garai et al. '06- they refer to the isotope data. But they do not mention the FTIR result- which happens to have given the exact opposite answer to what Garai et al. '06 found.

I suspect that their definition of "first" is similar to the President's definition of "victory", or Clinton's definition of "is".

Aside from a less on press release science, this is also a lesson on publishing a (vaguely) multidisciplinary study in the wrong field.

I suspect that most astronomy editors wouldn't have a clue as to who to pick as reviewers, and non-diamond people wouldn't know teh literature well enough to spot this.

If this had gone to a mineralogy journal, chances are this sort of basic mistake would have been caught in peer review, not post-wikipedia.