[ZenGum]

Direct from New Scientist:

Which nation's scientists are most prone to falsifying their results? A statistical debate has broken out this week, following the publication of a paper claiming that US researchers "are significantly more prone to engage in data fabrication or falsification than scientists from other countries".

To reach this conclusion, published in the Journal of Medical Ethics, Grant Steen of Medical Communication Consultants in Chapel Hill, North Carolina, searched the PubMed database of the biomedical literature for papers retracted between 2000 and 2010, recorded whether the retractions were due to error or fraud, and calculated a fraud-to-error ratio.

But as Richard Van Noorden pointed out on Nature's Great Beyond blog, this doesn't necessarily mean that US-based scientists engage in more fraud - they might simply publish fewer sloppy errors. By looking at total papers published, he calculated US researchers have to retract a smaller proportion because of fraud than those from China, India and South Korea.

Bob O'Hara, a statistician and ecologist at the University of Helsinki in Finland, then crunched the numbers on his Nature Network blog. The retraction rate due to fraud for US-based researchers was slightly above the norm, but nowhere near as high as in China (about three times above average) and India (five times). The differences between the US and its Asian competitors were highly statistically significant.

French researchers emerged as the least prone to fraud in this analysis. But as O'Hara explains, we shouldn't read too much into any of the figures, as there's a "missing data" problem: retractions only measure the rate at which people get caught out for fraud, not the rate at which it is committed.

Indeed, as New Scientist's long-running investigation into stem cell research at the University of Minnesota has shown, papers containing manipulated data may remain unchallenged for many years, until someone takes the time to pick them apart.

There are other biases to consider. Steen's paper confirms earlier findings that rates of retraction are higher in more prominent journals. That makes sense, as these papers will be subject to greater scrutiny.

In June, New Scientist found that US-based researchers were more successful than those elsewhere at getting their papers into high-impact journals in the hot field of cellular reprogramming. If that's also true in other areas of biomedical research, it may make US-based researchers seem more fraudulent than they actually are.

As I read this, the original study compared fraud-retractions to error-retractions.
A much better measure is retractions of either type compared to total publications. On that, the US comes out about normal.

So is the original study just badly done? or downright fraudulent?