Matters of fact

5 Mar 12
It used to be hard to gather all the information needed for monitoring and planning services. But now we’re in the Big Data Era, things are set to change

By John Thornton | 1 March 2012

It used to be hard to gather all the information needed for monitoring and planning services. But now we’re in the Big Data Era, things are set to change

Illustration: Natalie Wood

There is a lot of talk about ‘Big Data’ at the moment. But what is it and what are the implications for public sector bodies?

There was a time when it was difficult to get hold of the information needed for monitoring and planning. We used to build specialist management information systems to extract and compile data around a relatively small number of performance indicators. We are now entering what has been dubbed the Big Data Era, where customer relationship management systems and business processes produce huge amounts of data as by-products of day-to-day operations.

We are also beginning to collect much more geospatial information, feedback from social media interactions and real-time sensory data in areas such as energy conservation, telecare and traffic management. The result is that we are in danger of being overwhelmed by data.

What is different about this new environment is that we are now gathering information that we might or might not need. We don’t know what we don’t know – we don’t know the questions that might be posed in the future and we most likely do not understand the potential cause-and-effect relationships between the different types of data collected. But storage is relatively cheap and so we have begun to build huge virtual warehouses of potentially valuable data.

Why is this important for the public sector? The public services almost certainly produce more data than any other industry, plus the analytical perspectives are probably more diverse than in any other sector. In a period of continuing economic uncertainty, public service bodies – like other industries – need to focus on cost control, understanding customers, optimising operations and managing risk. 

Additionally, they need to be aware of the political and regulatory impacts of plans and policy changes.Take, for example, the recent rows over government benefit caps and Westminster council’s planned parking restrictions. In both these cases, assessing and modelling the political and regulatory effects of these decisions were probably as important, if not more so, than the assessment of their impact on revenues, costs and operations.

This means that we need a new mind-set, new tools and possibly new skill-sets. The tools are already becoming available and cloud computing is providing access to huge amounts of relatively cheap computing power and storage.

So we now need people who are capable of looking at the bigger picture, understanding the different perspectives and finding the relevant data to model options and impacts. This means that today’s accountants and definitely the next generation need strong information management skills with the ability to extract meaningful information from large data sets. They will also need to present evidence-based advice to boards and politicians.

We will probably see rising demand for ‘data scientists’ with strong statistical skills and the ability to understand the algorithms and analytical models that support scenario planning.

I think it was the Chinese military strategist, Sun Tzu, who wrote: ‘Strategy without tactics is the slowest route to victory. Tactics without strategy is the noise before defeat.’  The broadening and deepening of our information resources, coupled with the accelerated timescales for planning and reacting, mean that we should be better placed to assess and identify the strategies that will have the most significant business impact – and be better placed tactically to react quickly and decisively to changing circumstances.

I am not suggesting that software and information alone will deliver superior planning and outcomes. Good managers and leaders make better decisions and react more quickly if they have the necessary information available.  A key part of the skill of being a good manager or leader is knowing the right questions to ask.
Are you and your organisation ready for the era of Big Data?

John Thornton is director of e-ssential Resources and an independent adviser and writer on business transformation, financial management and innovation

CIPFA logo

Did you enjoy this article?