On the records

Public records, open data and Freedom of Information

That might be changing. The British government and a few others including Canada, Costa Rica, Colombia and Paraguay are working towards opening up the information about public procurement through an ‘open contracting data standard’.

The first version is launched today and sets out a technical standard for the data and documents that ought to be published at each stage of the contracting process, from the invitation to tender through to completion. Read more

Trust in institutions to use data is much lower than trust in them in general, according to a new survey for the Royal Statistical Society.

The poll of just over two thousand British adults carried out by Ipsos MORI found that the media, internet services such as social media and search engines and telecommunication companies were the least trusted to use personal data appropriately. Read more

Chris Cook

A few weeks ago, the Financial Times leader column – the voice of the newspaper – issued a fairly damning verdict on the Information Commissioner’s Office about its weak enforcement of the Freedom of Information Act. Big central government departments – particularly the Cabinet Office – routinely flout the law. The ‘paper wrote:

The Cabinet Office scarcely pretends to comply. And why would it? Christopher Graham, the Information Commissioner, is responsible for enforcing the act – and he has proved to be a paper tiger. He has the power to investigate, demand documents and prosecute the non-compliant. But his preferred regulatory tool is forbearance. So the worst that departments have to fear from refusing to follow the law is being asked to have another go at answering requests at some later date.

Mr Graham replied fairly grumpily on our letters page.

…we served a decision notice against the Department for Education, clearly setting out our position in general and on the specific point that the secretary of state’s private email was caught by the act (in the circumstances of that case). We also served a further decision notice ordering the Cabinet Office to comply with a request for information relating to the prime minister’s use of non-GSI email accounts. Non-compliance with the commissioner’s decision notice is contempt of court.

The record shows that the Information Commissioner’s Office regularly makes difficult decisions that challenge Whitehall – and we are not afraid to make them

It was not terribly convincing – and was filleted by others fairly comprehensively. But last week, the Information Commissioner’s Office released some documents in response to a FoIA request about the letter. The surprising part of the release is that it suggests the ICO’s staff not only wrote the letter, but are convinced by their arguments. Given that, it is worth unpacking the claims it makes – not least for their benefit. Read more

Chris Cook

The Freedom of Information Act has a clause which allows public authorities to ignore a request for information “if the request is vexatious”. It says little about what members of the public can do if they encounter vexatious government departments.

The Cabinet Office – which includes the prime minister’s office – seems now to be openly refusing to comply with the transparency law, especially when it comes to the person of the prime minister. His courtiers seem to regard FoIA requests as lèse-majesté, and to see fighting transparency as part of their roles. Read more

Martin Stabe

The London Fire Brigade has published its incident data for the past four years in the London Data Store, the capital’s open data repository.

The data, originally obtained by the Financial Times under the Freedom of Information Act and published last week in our London Fire Brigade response times map, is being released as open data because of widespread interest in the potential impact of fire station closures.

The London Fire Authority is defying mayor Boris Johnson’s order to put the proposed cuts to public consultation. The £45m cut in the brigade’s budget over two years, would see 12 fire stations, 18 engines and 520 jobs go. Read more

Chris Cook

Last week, I went to Wolverhampton where I spoke at a local debate, organised by the university and Pat McFadden, the local MP, about the local authority’s school. I was the warm-up act for Lord Adonis, former schools minister, setting the scene about the city’s education system before his talk on lessons on school improvement.

It was interesting event – and the city is clearly considering its future and the role of education within it. There is – judging by my inbox – serious and deep interest in improving schools in the city. One of the things I sought to do was set out Wolvo’s position in relation to the rest of the country – and what statistics about the city tell us.

Here is my presentation: Read more

Kate Allen

By Kate Allen and Jonathan Moules

A new institution which aims to maximise the UK’s world-leading position in the emerging field of data use for the creation of new businesses and services is set to launch today.

The Open Data Institute, which was announced by George Osborne in last year’s Autumn Statement, will be the first such organisation of its kind in the world. It will receive £10m government funding over the next five years and aims to raise an equal amount from private donors. It has already attracted $750,000 from eBay founder and philanthropist Pierre Omidyar. Read more

Kate Allen

The government has made admirable moves towards openness and transparency in recent years, with one landmark step being to publish all payments by government departments of £500 or more. Communities secretary Eric Pickles has gone further, this week deciding to start publishing all spending by his department of £250 and above.

But there is a problem. The spreadsheets list each individual payment, not total amounts per supplier. And suppliers’ VAT numbers are not provided. This makes summarising the payments – often across thousands of lines of data – a very difficult job for anyone aspiring to create aggregate figures.

Let’s take Mr Pickles’ department, Communities & Local Government, as an example. Its latest quarterly release contains 9,750 lines of data. It encompasses all sorts of activities, from PFI grants to mobile phone line rental charges. Many of the payments require an expert jargon-buster to interpret: did you know that “NNDR billing authorities receipts” means business tax?

The smallest payment listed during the period was £1.24 to Banner Business Supplies for stationery. The largest was £1.9bn to the Office of the Paymaster General (those NNDR billing authorities receipts again). A government department is a very complicated entity and trying to understand its activities through scrutinising its history of individual payments is rather like trying to write a biography of a subject by looking solely at their bank statements. Read more

The coalition’s drive to open access to official data was welcome, but  more needed to be done to make information comprehensible, according to a report by the House of Commons’ public accounts committee, which monitors the effectiveness of public spending.

Official willingness to publish data was not enough, the committee said, since the information provided could be rather impenetrable. Some of the data are published as very large files that cannot be opened using a conventional home computer. Other files are difficult to interpret or can be only understood with the aid of large glossaries.

“It is simply not good enough to dump large quantities of raw data into the public domain, “ said Margaret Hodge, who chairs the committee. “[Data] must be accessible, relevant and easy for us all to understand. Otherwise the public cannot use them.”

 Read more

Chris Cook

What measures should we use for spotting schools that are effective at helping poor children? Not the one proposed by the Department for Education. Read more

The House of Lords authorities are refusing to hand over officials’ estimates of how much it will cost taxpayers to replace the chamber with a mostly elected senate, prompting anger from Tory politicians.

Officials have rejected a freedom of information request by the Financial Times, saying that the relevant information was produced “solely” for the joint committee on Lords reform. “A decision was taken by them not to publish it as part of their report,” they said in their response.

David Davis, MP for Haltemprice and Howden, said there was a “clear-cut case” for the cost estimates to be put in the public domain.

 Read more

Chris Cook

Your birthday matters: children who are older when they start school as 4 year-olds outperform their peers. This is not a small effect, nor does it peter out as they get older. We can spot it easily at the national level among 16 year-olds. Read more

Chris Cook

The social mobility problem is not that there is a small number of weak schools serving a lot of poor kids. It is that poor children do badly in the majority of England’s schools. Read more

Chris Cook

A fortnight ago, MPs caught a fleeting glimpse of a process that has, to this point, taken place discreetly: the Information Commissioner’s Office investigation into the office of Michael Gove over suspected breaches of the Freedom of Information Act.

A transcript is now available for Mr Gove’s appearance before the education select committee, when he answered questions on the topic. He said the DfE had not released data from one document in response to FoI requests because it was political.

The law is straightforward: only government data is covered by the FoI Act. Party political or private business is never captured, even if it is sent via a government email address. Official business, however it is transmitted, is always covered.

In circumstances where there is a mix of party, personal and government business, official data is released and the remainder is redacted. So the whole text would need to be party political and not official for the document not to be covered by the act.

We have published it below. Read more

The release this morning of data detailing every Whitehall payment above £25,000 is a step towards the culture of public transparency that the previous Government intended to create when it passed the Freedom of Information Act a decade ago.

Rather than waiting for requests under the FOI regime, the coalition Government has committed to releasing this basic spending information proactively, in a format that allows scrutiny by anyone with the necessary time, software and skills.

This morning’s data release will be repeated each month, and from January, local government bodies will have to release similar datasets accounting for all transactions above £500.

Some of the frustrations with analysing public data to which journalists have become accustomed were absent. There were no files released as locked PDF documents that are difficult to import into database software, for example.

While each department’s monthly spending was released as a separate spreadsheet document, these were formatted in a consistent structure across departments, thanks to detailed Treasury guidance on how to release the data.

Nevertheless, analysing the data still posed significant technical challenges.

 Read more