I Think You'll Find It's a Bit More Complicated Than That Read Free

bring you satisfaction.
    2014

Being Wrong
    Guardian , 15 July 2011
    Morons often like to claim that their truth has been suppressed: that they are like Galileo, a noble outsider fighting the rigid and political domain of the scientific literature, which resists every challenge to orthodoxy.
    Like many claims, this is something for which it’s possible to gather data.
    Firstly, there are individual anecdotes that demonstrate the routine humdrum of medical fact being overturned. We used to think that hormone-replacement therapy significantly reduced the risk of heart attacks, for example, because this was the finding of several large observational studies. That kind of research has important limitations: if you just grab some women who are receiving prescriptions for HRT from their doctors, and compare them to women who aren’t getting HRT, you might well find that the women on HRT are healthier, but that might simply be because they were healthier to start with. Women on HRT might be richer, or more interested in their health, for example. At the time, this research represented our best guess, and that’s often all you have to work with. Eventually, after decades of HRT being widely used, a large randomised trial was conducted: they took 16,000 women who were eligible for HRT, and randomly assigned them to receive either real hormones, or a dummy placebo pill. At long last we had a fair test, and after several years of treatment had passed, in 2002, the answer fell out. HRT increased the risk of having a heart attack by 29 per cent.
    Were these findings suppressed? No. They were greeted eagerly, and with some horror: in fact, the finding was so concerning that the trial had to be stopped early, to avoid putting any further participants at risk, and medical practice was overturned.
    Even the supposed stories of outright medical intransigence turn out, on close examination, to be pretty weak: people claim that doctors were slow to embrace Helicobacter pylori as the cause of gastric ulcers, when in reality it only took a decade from the first murmur of a research finding to international guidelines recommending antibiotic treatment for all patients with ulcers.
    But individual stories aren’t enough. This week Vinay Prasad and colleagues published a fascinating piece of research about research . They took all 212 academic papers published in the New England Journal of Medicine during 2009. Of those, 124 made some kind of claim about whether a treatment worked or not. Then, they set about measuring how those findings fitted into what was already known. Two reviewers assessed whether the results in each study were positive or negative, and finally – separately – they decided whether these new findings overturned previous research.
    Seventy-three of the studies looked at new treatments, so there was nothing to overturn. But the remaining fifty-one were very interesting, because they were, essentially, evenly split: sixteen upheld a current practice as beneficial; nineteen were inconclusive; and, crucially, sixteen found that a practice believed to be effective was in fact ineffective, or vice versa.
    Is this unexpected? Not at all. If you like, you can look at the same problem from the opposite end of the telescope. In 2005, John Ioannidis gathered together all the major clinical research papers published in three prominent medical journals between 1990 and 2003; specifically, he took the ‘citation classics’, the forty-nine studies that were cited more than a thousand times by subsequent academic papers.
    Then he checked to see whether their findings had stood the test of time, by conducting a systematic search in the literature to make sure he was consistent in finding subsequent data. Of his forty-nine citation classics, forty-five had found that an intervention was effective, but in the time that had passed subsequently, only half of these findings had been positively replicated. Seven studies – 16 per cent of the