Big Data

Due to a mix of scarce resources, methodological difficulties and poor incentives, much of the data collected on the world’s poor is either inaccurate or missing. That’s one of the findings of a study being released today by the Overseas Development Institute (ODI) at the Cartagena Data Festival in Colombia.

While the trend of falling poverty is genuine, the ODI think that numbers in poverty may be being understated by up to 350 million, more than the entire population of the US. Read more

It is said that money makes the world go round, but in football’s transfer market we can be a little more specific on that money’s provenance. Our interactive graphic explores the net spend of the 265 clubs in world football to have each spent £10m or more in total since 1980, grouped by country.

Any guesses on the identity of the country whose clubs spend the most? Read more

The UK’s information technology sector could be about 40 per cent bigger than previously though, with at least 70,000 more ICT companies in operation.

That’s according to the National Institute of Economic and Social Research who have come up with a pretty novel way of measuring the size of Britain’s tech sector using one of the industry’s most hyped concepts ‘big data’.

 Read more

OK Cupid, an online dating site, has caused a bit of a stir recently about performing experiments on their users. But even without the ethical questions there’s reason to be skeptical about what their data can actually tell us.

Big Data, the book by Viktor Mayer-Schonberger and Kenneth Cukier, talks about two phenomena they believe will drive a big data revolution: ‘Digital exhaust’ and ‘N equals all’. The first refers to the trail of information we leave behind when using the internet that are the residue of clicks and typing. Read more

Trust in institutions to use data is much lower than trust in them in general, according to a new survey for the Royal Statistical Society.

The poll of just over two thousand British adults carried out by Ipsos MORI found that the media, internet services such as social media and search engines and telecommunication companies were the least trusted to use personal data appropriately. Read more

Emily Cadman

Can social media data give useful real-time information during the progress of natural disasters? A group of academics thinks it could.

Researchers from Warwick University in the UK looked at pictures uploaded to Flickr during the lead-up and aftermath of Hurricane Sandy. As the two graphs below show, there was a strong correlation between the number of pictures taken and atmospheric pressure.

 Read more

The rise in US food prices accelerated in September and has continued into October, according to a detailed study of retail prices – though the US federal government shutdown has robbed financial markets of any official measures of the state of the economy.

These unofficial inflation figures, from a US start-up called Premise, highlight the growing use of massive data collection and analysis – known as “big data” – to supplement and in some cases replace official economic statistics.

 Read more