Ben Goldacre has been looking at the claims of Government ministers that the evidence supports the case for their radical reforms of the NHS. John Burstow responded to Goldacres first article by writing a letter to the Guardain to which Goldacre just responded.
Here's what Goldacre says:
DON'T CHERRY-PICK NHS FINDINGS MINISTER
Last week we saw that the government had overstated the failings of the NHS by using dodgy figures (to be precise, they used misleading static figures instead of time trends). We saw that the health secretary Andrew Lansley's repeated claim that his reforms are justified by evidence was untrue: the evidence doesn't show that his price-based competition improves outcomes (if anything it makes things worse); and the evidence also doesn't show that GP consortiums improve outcomes (unless you cherry-pick only the positive findings).
It's OK if your reforms aren't supported by existing evidence: you just shouldn't claim that they are.
Now Lansley's junior minister, Paul Burstow, has kindly responded via the Guardian's letters page, repeating the same mistakes again, only more clumsily. I find this, in all seriousness, genuinely frightening from a minister, so I'll explain how he does it.
The government initially claimed that UK heart attack death rates were twice as bad as France. This was an overstatement: they are, but following recent interventions, the gap is closing so rapidly that on current trends it will have disappeared entirely by 2012. In response, Burstow cites a 2008 paper by McKee and Nolte which he says "concluded that the UK had one of the worst rates of mortality amenable to healthcare among rich nations".
Burstow either misunderstands or misrepresents this very simple and brief paper. It is a study explicitly looking at time trends, not static figures, and it once again finds that comparing 2003 with 1998, the UK still had fairly high rates of avoidable mortality, but these were falling faster than in all but one of the other 18 industrialised countries they examined. Meanwhile in the US, avoidable mortality improved at a disastrously slow pace, although they spent more money.
This is a paper showing the success of the NHS, and the fact that we are discussing such a huge improvement in avoidable mortality from Labour's first term in government is not my choosing: this is the paper that was cited by the Tory minister as evidence, bizarrely, of the NHS's recent failures.
Next Burstow says I "overlooked the impact assessment we published alongside the health and social care bill, where we present a thorough analysis of the evidence for and against our plans … studies show that GP fundholding and practice-based commissioning delivered shorter waits and fewer referrals to hospitals for patients".
You won't be surprised to hear studies show no such thing. If anything they show the opposite. Goldacre continues here. The article contains a link to Burstow's letter.
Gove's education reforms are similarly based on dodgy use of PISA stats.
Gove:
"Last week we discovered that we had fallen from 4th to 14th in the international league tables for science; 7th to 17th for reading, and 8th to 24th for maths.
How does the secretary of state explain how we were in the top 10 for all these subjects when the children sitting the tests had the majority of their education under a Conservative government, but had fallen down the rankings, relegated to the second division, when those sitting the tests had all their education under a Labour government."
Michael Gove, shadow children's secretary, House of Commons,11 December 2007
As FactCheck at Channel 4 news points out: "the researchers from the OECD say that the results do not show any evidence of a real decline in standards." "...So a man of Gove's legendary intelligence really has no excuse for trotting out these obviously misleading stats one more time." Go here. Gove's use of stats etc. to support Swedish style free schools also involves manipulation of data (e.g. ignoring recent poor Swedish performance, ignoring the performance of Finland).
Of course Labour could also use stats and evidence misleadingingly - e.g. those weapons of mass destruction.
Tories are wheeling in major reforms in health and education not because the evidence clearly supports such reforms, but because they are ideologically committed to such reforms. "We're not advocating reform for the sake of ideology" says Burstow in his response to Goldacre. Right.
Here's what Goldacre says:
DON'T CHERRY-PICK NHS FINDINGS MINISTER
Last week we saw that the government had overstated the failings of the NHS by using dodgy figures (to be precise, they used misleading static figures instead of time trends). We saw that the health secretary Andrew Lansley's repeated claim that his reforms are justified by evidence was untrue: the evidence doesn't show that his price-based competition improves outcomes (if anything it makes things worse); and the evidence also doesn't show that GP consortiums improve outcomes (unless you cherry-pick only the positive findings).
It's OK if your reforms aren't supported by existing evidence: you just shouldn't claim that they are.
Now Lansley's junior minister, Paul Burstow, has kindly responded via the Guardian's letters page, repeating the same mistakes again, only more clumsily. I find this, in all seriousness, genuinely frightening from a minister, so I'll explain how he does it.
The government initially claimed that UK heart attack death rates were twice as bad as France. This was an overstatement: they are, but following recent interventions, the gap is closing so rapidly that on current trends it will have disappeared entirely by 2012. In response, Burstow cites a 2008 paper by McKee and Nolte which he says "concluded that the UK had one of the worst rates of mortality amenable to healthcare among rich nations".
Burstow either misunderstands or misrepresents this very simple and brief paper. It is a study explicitly looking at time trends, not static figures, and it once again finds that comparing 2003 with 1998, the UK still had fairly high rates of avoidable mortality, but these were falling faster than in all but one of the other 18 industrialised countries they examined. Meanwhile in the US, avoidable mortality improved at a disastrously slow pace, although they spent more money.
This is a paper showing the success of the NHS, and the fact that we are discussing such a huge improvement in avoidable mortality from Labour's first term in government is not my choosing: this is the paper that was cited by the Tory minister as evidence, bizarrely, of the NHS's recent failures.
Next Burstow says I "overlooked the impact assessment we published alongside the health and social care bill, where we present a thorough analysis of the evidence for and against our plans … studies show that GP fundholding and practice-based commissioning delivered shorter waits and fewer referrals to hospitals for patients".
You won't be surprised to hear studies show no such thing. If anything they show the opposite. Goldacre continues here. The article contains a link to Burstow's letter.
Gove's education reforms are similarly based on dodgy use of PISA stats.
Gove:
"Last week we discovered that we had fallen from 4th to 14th in the international league tables for science; 7th to 17th for reading, and 8th to 24th for maths.
How does the secretary of state explain how we were in the top 10 for all these subjects when the children sitting the tests had the majority of their education under a Conservative government, but had fallen down the rankings, relegated to the second division, when those sitting the tests had all their education under a Labour government."
Michael Gove, shadow children's secretary, House of Commons,11 December 2007
As FactCheck at Channel 4 news points out: "the researchers from the OECD say that the results do not show any evidence of a real decline in standards." "...So a man of Gove's legendary intelligence really has no excuse for trotting out these obviously misleading stats one more time." Go here. Gove's use of stats etc. to support Swedish style free schools also involves manipulation of data (e.g. ignoring recent poor Swedish performance, ignoring the performance of Finland).
Of course Labour could also use stats and evidence misleadingingly - e.g. those weapons of mass destruction.
Tories are wheeling in major reforms in health and education not because the evidence clearly supports such reforms, but because they are ideologically committed to such reforms. "We're not advocating reform for the sake of ideology" says Burstow in his response to Goldacre. Right.
Comments
But they way, if you've not read "Bad Science" (the book), do so today.