Last month, in advance of a report on 14 hospital trusts with relatively high death rates, it was widely reported that there had been 13,000 “unnecessary” deaths.
But, as a leading academic points out in today’s British Medical Journal, this demonstrates an epic lack of understanding of the concept of an average.
The 13,000 figure is the difference between the actual number of deaths in the 14 hospitals compared to the “expected” level. But what some writers failed to understand is that the “expected” level equated to the national average. As a result much of the coverage was seriously misleading, according to David Spiegelhalter, Winton professor for the public understanding of risk at the University of Cambridge. Read more
At the moment, the Department for Education is considering changes to the league tables and the exam system. This seems an opportune moment to make a simple point about qualification-awarding and accountability: English school examinations are subject to measurement error in a really big way.
Here is a simple thought experiment to flesh it out. Imagine a class of 100 students. Let us specify that each one has a “true” ability that means that one pupil should get one point, one pupil should get two, one should get three and so on – up to 100 marks. Now, let’s award this class one of 10 grades: 90+ gets you an A, 80+ a B and so on.
Let us assume that the tests are perfect. If that were the case, you would get ten individuals in each grade. Easy enough. But what happens if we start introducing errors into the test? We can do that with a set of exotically named (but very simple) “Monte Carlo” estimates, which I calculated using this simple spreadsheet. Read more
Last week, I went to Wolverhampton where I spoke at a local debate, organised by the university and Pat McFadden, the local MP, about the local authority’s school. I was the warm-up act for Lord Adonis, former schools minister, setting the scene about the city’s education system before his talk on lessons on school improvement.
It was interesting event – and the city is clearly considering its future and the role of education within it. There is – judging by my inbox – serious and deep interest in improving schools in the city. One of the things I sought to do was set out Wolvo’s position in relation to the rest of the country – and what statistics about the city tell us.
Here is my presentation: Read more
If you’re going to have a bet this party conference season, take on an MP – that’s the obvious conclusion from an Ipsos Mori poll for the Royal Statistical Society, which suggests that our elected representatives are simply asking to be fleeced.
As part of its Parliamentary lobbying work, the RSS set a basic probability question for nearly 100 MPs. ‘If you spin a coin twice, what is the probability of getting two heads?’*
More than half of them got the answer wrong.