Monday, September 17, 2007

Publish or Perish Spreads to Science

The Wall Street Journal has a piece on the shoddy standards that the avalanche of "scientific" studies has engendered.
Statistically speaking, science suffers from an excess of significance. Overeager researchers often tinker too much with the statistical variables of their analysis to coax any meaningful insight from their data sets. "People are messing around with the data to find anything that seems significant, to show they have found something that is new and unusual," Dr. Ioannidis said.

Why this is so becomes immediately clear. Follow the money.
In the U. S., research is a $55-billion-a-year enterprise that stakes its credibility on the reliability of evidence and the work of Dr. Ioannidis strikes a raw nerve. In fact, his 2005 essay "Why Most Published Research Findings Are False" remains the most downloaded technical paper that the journal PLoS Medicine has ever published.

Perhaps Dr. Ioannidis is a new Richard Feynman who in his prescient message to CalTech grads about "Cargo Cult Science," predicted the very quandry of second-rate "scientists" trying to tease cash out of huge government and NGOs, many of whom have an agenda. Feynman:
Nature's phenomena will agree or they'll disagree with your theory. And, although you may gain some temporary fame and excitement, you will not gain a good reputation as a scientist if you haven't tried to be very careful in this kind of work. And it's this type of integrity, this kind of care not to fool yourself, that is missing to a large extent in much of the research in cargo cult science.

Feynman had predicted catastrophic failure in one out of fifty Shuttle missions, contrasting with NASA's prediction of one in a million. Two catastrophes in little over one hundred missions. Just who was closer to being right?
Of course, NASA's "chief climatologist" has been on a crusade for two decades against what his "research," flawed even more than the spurious NASA numbers on catastrophic Shuttle Missions, indicates is Anthropogenic Global Warming. His chief missionaries are media and political hucksters like Al Gore and Leonardo Di Caprio, both lacking in academicide credentials---though that is not unusual nowadays with sententious public posturing over subjects that are trendy and PC, but lacking in substantial credence, including the UN cattle call IRCC pronunciamento last Spring.Dr. Ioannides:
Every new fact discovered through experiment represents a foothold in the unknown. In a wilderness of knowledge, it can be difficult to distinguish error from fraud, sloppiness from deception, eagerness from greed or, increasingly, scientific conviction from partisan passion. As scientific findings become fodder for political policy wars over matters from stem-cell research to global warming, even trivial errors and corrections can have larger consequences.

And a bit later, the article notes that peer-review can't catch all the errors:
To root out mistakes, scientists rely on each other to be vigilant. Even so, findings too rarely are checked by others or independently replicated. Retractions, while more common, are still relatively infrequent. Findings that have been refuted can linger in the scientific literature for years to be cited unwittingly by other researchers, compounding the errors.

Stung by frauds in physics, biology and medicine, research journals recently adopted more stringent safeguards to protect at least against deliberate fabrication of data. But it is hard to admit even honest error. Last month, the Chinese government proposed a new law to allow its scientists to admit failures without penalty. Next week, the first world conference on research integrity convenes in Lisbon.

Overall, technical reviewers are hard-pressed to detect every anomaly. On average, researchers submit about 12,000 papers annually just to the weekly peer-reviewed journal Science. Last year, four papers in Science were retracted. A dozen others were corrected.

The Korean scientist's controversy subsides from memory and the deluge continues with dozens of scientists probably cooking up experiments and tweaking stats to arrive at a Cargo Cult Moment. In the end, Global Warming may go the way of Global Cooling, all of thirty years ago. Back when Al Gore claims he had an epiphany about Global Warming [I wonder what that magic moment was really about. Maybe Barnum & Bailey?]
The punchline of Dr. Ioannidis' research is not pleasant.
No one actually knows how many incorrect research reports remain unchallenged.

Earlier this year, informatics expert Murat Cokol and his colleagues at Columbia University sorted through 9.4 million research papers at the U.S. National Library of Medicine published from 1950 through 2004 in 4,000 journals. By raw count, just 596 had been formally retracted, Dr. Cokol reported.

Informatics is a new field floating into my ken. But finally, the ultimate last word from the science community and then I will drift into salutary quotes from one of my all time favorite reads even more apropos in this time of feverish overstatement.
"The correction isn't the ultimate truth either," Prof. Kevles said.

Extraordinary Delusions and the Madness of Crowds is online in its entirety, and I would commend the gentle reader if he simply clicks the link. Here's the first line in English:
"In reading the history of nations, we find that, like individuals, they have their whims and their peculiarities; their seasons of excitement and recklessness, when they care not what they do. We find that whole communities suddenly fix their minds upon one object, and go mad in its pursuit; that millions of people become simultaneously impressed with one delusion, and run after it, till their attention is caught by some new folly more captivating than the first."

Yes, it does read a bit like Gibbons Decline and Fall of the Roman Empire. But come to think of it, the delirious supporters of Global Warming hysteria have much the same agenda [Decline and Fall] for the United States economic system. And once again, it may be religion, this time the new religion of science Feynman warned us against, snowe-balling us into Global Warming legislative gymnastics.

No comments :