Science Fictions

...betrayal of public trust is perhaps the most pernicious thing about fraud. People have invested a great deal in science, up to and including the health of their children. Fraudsters make a mockery of that trust. Although the fraudsters themselves are obviously to blame, our scientific system also deserves condemnation. Not only do the most glamorous journals encourage people to send only the flashiest findings – more or less guaranteeing that some small fraction of scientists will turn to deception to achieve such flashiness – but journal editors often act with reluctance and recalcitrance when even quite solid evidence of wrongdoing comes to light.¹⁴⁵

When it comes to studies of such extraordinarily complex systems as the body or the brain, or an ecosystem, the economy or society, it’s rare for scientists to find one factor that has a massive effect on another. Instead, most of the psychological, social and even medical phenomena we’re interested in are made up of lots of small effects, each of them playing a small role. For example, if economists want to explain why different people in their sample have different incomes, they’d need to take into account where participants live, their family backgrounds, their abilities, personalities and education, their country’s tax system and its changes over time, and a whole host of other factors and experiences that might have nudged the participants’ fortunes in one direction or another throughout their lives. The fact that small effects are so much more common and, collectively, so much more influential than big ones makes underpowered studies, which portray a world full of large effects, all the more misleading.

In an age of ‘churnalism’, where time-pressed journalists often simply repeat the content of press releases in their articles (science news reports are often worded virtually identically to a press release), scientists have a great deal of power – and a great deal of responsibility.²⁸ The constraints of peer review, lax as they might be, aren’t present at all when engaging with the media, and scientists’ biases about the importance of their results can emerge unchecked. Frustratingly, once the hype bubble has been inflated by a press release, it’s difficult to burst. A study in 2017 found that only around 50 per cent of health studies covered in the media are eventually confirmed by meta-analyses (that is, 50 per cent are found to be broadly replicable). This finding would be scandalous enough on its own, but what makes it even worse is the fact that those meta-analyses are rarely, if ever, covered in the press.²⁹ By that time the damage might already have been done. With apologies to Jonathan Swift: hyped science flies, and the refutations come limping after it.