Tamiflu: an analysis of all the data

These drugs were handed out via a phoneline during the swine flu pandemic of 2009 as part of a wider public health strategy.

Professor Carl Heneghan of Oxford University's Department of Primary Care Health Sciences and colleagues in the independent Cochrane Collaboration are clear that the money was wasted. They argue that the decision to stockpile the drugs might have been different had we had access to all the clinical data on their effectiveness.

Now we do have that evidence, and Carl says: 'There is no credible way these drugs could prevent a pandemic.' Speaking at a media briefing at the Science Media Centre in London, he said the money spent on stockpiling had been 'thrown down the drain'.

Since 2009, the Cochrane researchers have had a long running battle with the drug firms that manufacture Tamiflu and Relenza (Roche and GSK, respectively) to get unconditional access to their full data. They finally received everything last year, after first GSK then Roche said they would provide the materials – a significant development in the campaign to increase openness and accessibility of complete trial data.

The Cochrane group has been significant players, along with the AllTrials campaign, the BMJ medical journal, Ben Goldacre and others, in changing the whole approach to this issue among researchers, journals, drug firms and regulators. The simple argument is that if we are to make the right decision on what are the best drugs – considering their safety, effectiveness and the balance of benefits they offer in treating conditions over their side-effects – we need to have all the evidence available.

The researchers have now made that assessment for Tamiflu in the prevention and treatment of flu. They have reviewed a phenomenal amount of material, and with the BMJ and the Cochrane Collaboration, have published their conclusions today. They call on government and health policy decision makers to review guidance on the use of Tamiflu in light of their new evidence.

They found that Tamiflu is effective – but it shortens symptoms of flu by only around half a day on average. And importantly, they say, there is no good evidence to support claims that it reduces complications of influenza or admissions to hospital.

Then there are the side effects. Using Tamiflu to treat flu, the evidence confirms an increased risk of suffering from nausea and vomiting.

When Tamiflu is used to prevent flu, the drug can reduce the risk of people suffering symptomatic influenza. But there was an increased risk of headaches, psychiatric disturbances, and kidney events.

The review authors, Drs Tom Jefferson, Carl Heneghan and Peter Doshi, conclude that there are insufficient grounds to support the stockpiling of Tamiflu for mass use in a pandemic. From the best conducted randomised trials, there just isn’t enough evidence on the crucial elements of reducing serious complications of flu that can lead to hospitalisation and death, and the prevention of spread of flu. On the other hand we know there would be side-effects.

Not all scientists agree on the assessment of the balance of benefits of these antivirals versus their side-effects. Virologist Professor Wendy Barclay at Imperial College London believes the shorter time that symptoms last is important: 'Although one day does not sound like a lot, in a disease that lasts only 6 days, it is…We have only two drugs with which we can currently treat influenza patients and there is some data to suggest they can save lives. It would be awful if, in trying to make a point about the way clinical trials are conducted and reported, the review ended up discouraging doctors from using the only effective anti-influenza drugs we currently have.'

Roche, the manufacturers of Tamiflu, fundamentally disagree with the overall conclusions of the Cochrane review and criticised some of the report’s methodology. In media reports, UK Medical Director Dr Daniel Thurley has said: 'Roche stands behind the wealth of data for Tamiflu and the decisions of public health agencies worldwide, including the US and European Centres for Disease Control & Prevention and the World Health Organisation.'

Indeed, Roche have pointed to a large observational trial in the Lancet Respiratory Medicine that they funded which recently reported a reduction in deaths among those hospitalised with swine flu H1N1, though there are some who disagree with that analysis too.

So what to make of all of this? An editorial in the Guardian concludes: 'The only way to resolve the argument is proper science. That means transforming clinical trials, harmonising the way they are carried out. It has happened with malaria drugs, and it is happening with HIV. The industry must allow access to their data. Confident that like is compared with like, trials can then be subjected to meta-analysis, allowing statisticians to drill down into sub-populations to establish when a drug performs most effectively.'

The editorial points to the need to be able to react swiftly and carry out good research actually during pandemics, as former Oxford University professor and now director of the Wellcome Trust, Jeremy Farrar, argued in the paper last month.

What has really changed is the ability to have these discussions based on all of the evidence. There is a real shift in the level of scrutiny and the analyses that are now possible with access to all clinical trial data (although dealing with all these reams of data also brings new challenges too). That is a phenomenal change and a real achievement by the Cochrane researchers.

David Spiegelhalter, Winton Professor of the Public Understanding of Risk at the University of Cambridge, comments: 'This is a ground-breaking review. Since important studies have never been published, the reviewers have had to go back to clinical trial reports comprising over 100,000 pages: the effort to obtain these is a saga in itself. The poor quality of these reports clearly made extracting relevant data a massive struggle, with many pragmatic assumptions having to be made, but the final statistical methods are standard and have been used in hundreds of Cochrane reviews. Let’s hope that in future high-quality data can be routinely obtained and this type of review becomes unnecessary.'