We have the technology

26 Sep 14
There’s a lot of talk about ‘big data’ and what it can do for the public sector. The analytical software is available but making sense of data and acting more intelligently as a result are the big challenges

By John Thornton | 26 September 2014

There’s a lot of talk about ‘big data’ and what it can do for the public sector. The analytical software is available but making sense of data and acting more intelligently as a result are the big challenges

Smart thinking October

A health website initially developed by researchers at Boston Children’s Hospital in their spare time claims to have identified the Ebola outbreak in West Africa a week before it was officially identified by the World Health Organisation. HealthMap detects and tracks diseases by automatically collecting information from online news aggregators, social media and official public health reports to provide a comprehensive view of the development of infectious diseases. It then uses maps and timelines to present this information in a simple and easy to understand format. This is ‘big data’ in action and a great example of the benefits that can be derived quickly and cheaply from the wealth of data that is now available.

Big data is revolutionising planning and decision-making in both the public and private sectors. Customer relationship management systems and business processes are producing huge amounts of data as byproducts of day-to-day operations. We are also collecting more geospatial information, feedback from social media interactions and real time sensory data in areas like energy conservation, tele-care and traffic management. Making sense of the data and acting more intelligently as a result are the real challenges.

Analytical software can improve and often automate decisions and actions. You are probably on the receiving end of automated decisions every day, for example, when your credit card company judges whether it’s you using the card and when you receive targeted online adverts or recommendations from online retailers. These are easy transactional decisions and automated analytics offer huge opportunities for greater speed and efficiency across the public services.

The greater challenge is when we try to use big data to address thornier issues. In the 1950s a doctor in the emerging field of epidemiology, the study of patterns in disease, noticed the rising incidence of childhood cancers. Most diseases then correlated with poverty, but the children dying from cancer seemed to come mostly from more affluent families. 

The doctor, Alice Stewart, struggled to raise funding for research and realised that she had only one chance at collecting her data. She included in her questionnaire everything she could think of: did they have indoor or outdoor plumbing; did they eat fish and chips; did they consume fizzy drinks? The findings were clear; the mothers of the children who died had been X-rayed when pregnant. Stewart discovered that exposing a fetus to a single diagnostic X-ray doubled the risk of an early death from cancer. The findings were widely published and people got very excited.  

The problem was that the findings conflicted with the perceived wisdom. There was huge enthusiasm for using X-ray machines which were part of the  sexy new technology coming into medicine. There was a belief that X-rays were safe up to a point and doctors rejected the idea they could be directly harming patients. Plus, this was the 50s and Stewart was a woman.

Most people would have started to question their work. But Stewart worked with a statistician George Kneale, who tried to disprove her findings by using different models and crunching the numbers in different ways. He failed. However, it took another 25 years before hospitals in Britain and America stopped X-raying pregnant women. 

Most of us carry a lot of ‘baggage’ into the decision-making process and we don’t necessarily believe what the data is telling us. The data and the technology are available, but are we ready for big data? 

John Thornton is an independent adviser and writer on business transformation, financial management and innovation [email protected]

This feature was first published in the October edition of Public Finance magazine



Transparent

CIPFA logo

Did you enjoy this article?

AddToAny

Top