By David Walker
2 September 2010
Dreamt up by the Conservatives’ one-man think-tank, Oliver Letwin, the idea of militant (if seated) activists fuelled speeches by David Cameron and Francis Maude during and after the election. And now they have given the statistical squaddies their marching orders. Last month Communities Secretary Eric Pickles said that hand in hand with his proposed demolition of the Audit Commission went a plan ‘to unleash an army of armchair auditors’.
His ambition, he said earlier, was to make those doling out public money think twice as the massed ranks of second-guessers and internet observers hover over their shoulders. ‘Local people should be able to hold politicians and public bodies to account over how their hard-earned cash is being spent and decisions made on their behalf.’
Come January, councils will put items of spending above £500 online. Corby, Islington, Hammersmith have already made their spending visible, along with the Department for Communities and Local Government itself and its arm’s-length bodies. This, Letwin said before the election, would both keep public bodies honest, by letting the public inspect them, and apply pressure on their spending because the public would chivvy them and quiz individual items. The prospect the government holds out is ‘super audit’. Professional auditors want evidence of due process; the armchairs want to know why.
‘Ministers in this government are not going to be able to get away with all the waste,’ Prime Minister David Cameron said. Maybe ministers hope the armchairs will regard most public spending as waste and applaud the cuts, not just bagels for staff meetings and the proverbial pot plants.
But other ministers have not rushed to join Pickles. The Department of Health has not said whether GPs in the new consortiums that will replace primary care trusts would also be subject to the armchairs. Would GPs have to publish their spending online? The British Medical Association is not going to like that. And will the armchairs get to see what Capita or United Health spend, if such firms take over NHS local commissioning? Similarly, Education Secretary Michael Gove has been coy about academies and his projected parent-run schools: who gets a sight of their spending?
Private firms have not exactly shouted their approval of the armchair revolution. According to the Society of Information Technology Management, fuller data about contracts could lead to more competitive bids for government business by exposing the profit margins of incumbent contractors. Equally, this visibility ‘could make public sector work less attractive’, it said.
Perhaps as the government settles into power, its early enthusiasm for openness will fade. Not all Tories are convinced of the need for such transparency, either. On the backbenches sit MPs who once agreed with Sir James Goldsmith, father of Richmond Park MP Zac, who deplored the ‘see-through society’ as an invention of socialists.
The government still believes in a class of evidence different from what the armchairs could ever uncover – the evidence of rigorous surveys, academic investigation and disciplined inquiry. In a previous era, Tory governments cancelled social surveys because ministers such as Sir Keith Joseph believed more data would foment agitation for more public spending. But now, in the summer Comprehensive Spending Review battle, universities minister David Willetts is said to be fighting tooth and nail to secure funding for the 2012 cohort study. This is a big sample survey of children born on a given day, who are then followed throughout their life course. This is not just to redeem a promise Willetts made in opposition, but because he believes in the value of the data for policy.
Also battling through the Whitehall jungle is the chief scientific adviser John Beddington, who is asking whether departments use evidence enough. Another sign that the coalition government is not averse to numbers was Cabinet Office minister Francis Maude’s decision to go ahead with the 2011 Census, swallowing criticisms he made in opposition about its cost and intrusiveness. (In theory, you could ask the armchairs to count themselves.)
Civil servants have gone on publishing voluminous data on benefits, the conditions of poor families and crime. Although much of it is ‘legacy data’ from the Labour era, ministers still seem to believe in the need for objectivity. At the Department for Culture, Media & Sport, the national survey of culture, leisure and sport continues; and the Department for Transport collects reams of material on road and rail use, walking and cycling. While Work and Pensions Secretary Iain Duncan Smith has strong personal views about the power of marriage, he has not stopped publishing data others might say proves the contrary.
Last week, the government learnt the hard way the difference between data and interpretation, especially if it comes not from the armchair but from unimpeachable researchers. Given time to chew through the Budget books, the Institute for Fiscal Studies sharply criticised its impact on poor households. Credit the Treasury and Office for National Statistics for the richness of available data; but beware the first signs from the new government of blaming the analyst-messenger.
Will the government be tempted to cut? The ONS is biting its nails over the damage the Comprehensive Spending Review might do to Whitehall’s analytic capacities, and ominous noises are heard in some departments about the fate of social research. ‘This isn’t a government interested in evaluation,’ says a senior civil servant. ‘The government is just not commissioning assessments of its policy initiatives.’
Maude thinks pilots are fine in cockpits, but is not keen on paying for them in advance of rolling out prized Tory policies. His track record is mixed. The UK Statistics Authority had to send him a polite but firm letter of reprimand for publishing figures on civil service employment based on a ring round of departments by his special advisers. The ONS itself had been about to publish up-to-date and certified numbers.
Two glaring examples of evidence-less policy-making stand out from the early months. Gove ploughed ahead with parent-run schools without pausing for breath; Health Secretary Andrew Lansley produced his GP consortiums plan fully cooked from his back pocket. Both might have been given geographically delimited trials and both appear to be examples of the hasty policy-making for which the Opposition caned Labour a few years ago.
The government also ceased funding speed cameras without a review or appraisal of costs and benefits. Culling quangos might cut independent appraisal; ministers have seemed unconcerned. The National Policing Improvement Agency – its abolition announced in the summer – has been working on the effectiveness of CCTV. Now there is a subject armchair auditors might have strong views on. What if they clashed with the objective evidence put forth by the analysts?
The government has taken against polling and opinion research. Communities minister Grant Shapps declared the biennial Place Survey a ‘burden’ on councils. This was the postal survey of residents views conducted by councils over the past couple of years against a common template of questions, with the results put in league table form.
Wasteful, said Shapps. ‘These surveys are a cosmetic exercise which never change anything. The idea that council bureaucrats are forced to turn themselves into amateur pollsters to ask highly intrusive personal questions about their residents seems entirely out of place.’ Yet councils had emblazoned Place Survey results on their websites, presumably as an attested and objective statement of residents’ views.
It would be premature to conclude the coalition has ended ‘evidence for policy’, the slogan on which Labour came to power in 1997. Remember how, for a time, former prime minister Tony Blair and colleagues were so keen on evidence that they barely let projects such as Sure Start and the New Deal for Communities get going before asking academics to pull them up by the roots and examine the results.
But Labour was none too keen on evidence getting in the way of ministers’ fixed beliefs. Examples are previous justice secretary Jack Straw’s conviction that prison worked and that more police officers cut the rate of crime. Dispassionate analysis suggested Home Office action accounted for about a fifth of the reduction in crime on Labour’s watch, the rest was caused by better employment prospects, training and college courses for young people and, most banal of all, a drop in the number of young men in the age groups most likely to commit offences. And, in a pamphlet out in June, former education secretary Estelle Morris ruefully admitted ‘pressure for quick fixes can outweigh research when ministers set schools policy’.
This summer’s judgement has to be that the government values evidence up to a point, and evidence of a quality that the armchairs could never amass. Despite blaring headlines in the Daily Mail last week, ministers chose not to second-guess the decision by the National Institute for Health and Clinical Excellence not to endorse Avastin, a drug used in late-stage bowel cancer. The press quoted Barbara Moss from Worcester complaining that Nice had put a value on life and saying she was ‘living proof’ that Avastin worked.
It was a classic instance of mistaking individual for average outcomes. The Guardian columnist Ben Goldacre has made fun for a long time of statistical folly and ignorance on the part of public, press and ministers. And gingerly – because no-one likes to criticise democracy – voices have begun asking about the potential of the armchair auditors. As well as drugs that are not cost-effective, they might favour homeopathy, for example. This is still provided on the NHS although the present government, like its predecessor, says it believes the clinical evidence that the treatments do not work. But – as the NHS Choices Website says – ‘there are many considerations when making policy decisions, and patient choice is an important factor to consider’.
In the Truth is out there, published in March, the Audit Commission wondered: ‘What information will capture the public’s imagination and therefore be used? Putting large volumes of data online will not by itself be enough to enable and encourage people to exercise choice or influence public decision-making. With widespread concern about the trustworthiness of public information, what assurance will members of the public need in order to trust the information made available to them?’
So much depends on the quality of the data eyeballed by the internet mavens. Who is to assess the accuracy of what the armchairs behold? The government website data.gov.uk wants councils to use similar templates for the information they put out, which ought to make comparison easier.
A government committed to evidence might, in theory, have researched the prospect for armchair auditors and other dimensions of the Big Society before they became policies. What do we know about people’s enthusiasm and capacity? Ben Page of Ipsos Mori says his surveys imply a ‘seismic shift’ would be necessary to get the involvement the government envisages.
Gillian Fawcett, head of public sector at the Association of Chartered Certified Accountants, says: ‘The reality is that very few members of the public currently look at local authorities’ accounts – even though that opportunity is available to them. In many cases, where people are interested in accounts at all, their interest will be restricted to a specific issue, often to an area of personal interest.’
But so far the audit professionals are relaxed about the changes. CIPFA’s Ian Carruthers talks of mixing and matching the best of the old regime and the ‘potential benefits of the new framework’. Still, a nagging question remains: who might audit the armchairs, and subject their views to scrutiny? Pollster Bobby Duffy notes how people rely on the media in forming their perceptions and sometimes do not let data get in the way; they regularly misperceive reality, for example, on knife crime or teenage pregnancy. ‘The public often get it badly wrong, but few dare say so.’
David Walker is director of communications at the Audit Commission; these are his own views. His book with Polly Toynbee, The verdict: did Labour change Britain?, will be published by Granta in October
For full coverage of the abolition of the Audit Commission, click here
2 September 2010
So farewell then Audit Commission and hello amateur bean counters. David Walker takes a look at the coalition’s plans for ‘super audit’ – and wonders if they’re outside most people’s comfort zone
Who needs to pass the CIPFA exam? In this post-bureaucratic age, we’ve no need for inspectors and qualifications. Instead, ordinary people will sit at home, poring over the ledgers online, gesticulating wildly when they spot an anomaly – or even a spending item they don’t like.Dreamt up by the Conservatives’ one-man think-tank, Oliver Letwin, the idea of militant (if seated) activists fuelled speeches by David Cameron and Francis Maude during and after the election. And now they have given the statistical squaddies their marching orders. Last month Communities Secretary Eric Pickles said that hand in hand with his proposed demolition of the Audit Commission went a plan ‘to unleash an army of armchair auditors’.
His ambition, he said earlier, was to make those doling out public money think twice as the massed ranks of second-guessers and internet observers hover over their shoulders. ‘Local people should be able to hold politicians and public bodies to account over how their hard-earned cash is being spent and decisions made on their behalf.’
Come January, councils will put items of spending above £500 online. Corby, Islington, Hammersmith have already made their spending visible, along with the Department for Communities and Local Government itself and its arm’s-length bodies. This, Letwin said before the election, would both keep public bodies honest, by letting the public inspect them, and apply pressure on their spending because the public would chivvy them and quiz individual items. The prospect the government holds out is ‘super audit’. Professional auditors want evidence of due process; the armchairs want to know why.
‘Ministers in this government are not going to be able to get away with all the waste,’ Prime Minister David Cameron said. Maybe ministers hope the armchairs will regard most public spending as waste and applaud the cuts, not just bagels for staff meetings and the proverbial pot plants.
But other ministers have not rushed to join Pickles. The Department of Health has not said whether GPs in the new consortiums that will replace primary care trusts would also be subject to the armchairs. Would GPs have to publish their spending online? The British Medical Association is not going to like that. And will the armchairs get to see what Capita or United Health spend, if such firms take over NHS local commissioning? Similarly, Education Secretary Michael Gove has been coy about academies and his projected parent-run schools: who gets a sight of their spending?
Private firms have not exactly shouted their approval of the armchair revolution. According to the Society of Information Technology Management, fuller data about contracts could lead to more competitive bids for government business by exposing the profit margins of incumbent contractors. Equally, this visibility ‘could make public sector work less attractive’, it said.
Perhaps as the government settles into power, its early enthusiasm for openness will fade. Not all Tories are convinced of the need for such transparency, either. On the backbenches sit MPs who once agreed with Sir James Goldsmith, father of Richmond Park MP Zac, who deplored the ‘see-through society’ as an invention of socialists.
The government still believes in a class of evidence different from what the armchairs could ever uncover – the evidence of rigorous surveys, academic investigation and disciplined inquiry. In a previous era, Tory governments cancelled social surveys because ministers such as Sir Keith Joseph believed more data would foment agitation for more public spending. But now, in the summer Comprehensive Spending Review battle, universities minister David Willetts is said to be fighting tooth and nail to secure funding for the 2012 cohort study. This is a big sample survey of children born on a given day, who are then followed throughout their life course. This is not just to redeem a promise Willetts made in opposition, but because he believes in the value of the data for policy.
Also battling through the Whitehall jungle is the chief scientific adviser John Beddington, who is asking whether departments use evidence enough. Another sign that the coalition government is not averse to numbers was Cabinet Office minister Francis Maude’s decision to go ahead with the 2011 Census, swallowing criticisms he made in opposition about its cost and intrusiveness. (In theory, you could ask the armchairs to count themselves.)
Civil servants have gone on publishing voluminous data on benefits, the conditions of poor families and crime. Although much of it is ‘legacy data’ from the Labour era, ministers still seem to believe in the need for objectivity. At the Department for Culture, Media & Sport, the national survey of culture, leisure and sport continues; and the Department for Transport collects reams of material on road and rail use, walking and cycling. While Work and Pensions Secretary Iain Duncan Smith has strong personal views about the power of marriage, he has not stopped publishing data others might say proves the contrary.
Last week, the government learnt the hard way the difference between data and interpretation, especially if it comes not from the armchair but from unimpeachable researchers. Given time to chew through the Budget books, the Institute for Fiscal Studies sharply criticised its impact on poor households. Credit the Treasury and Office for National Statistics for the richness of available data; but beware the first signs from the new government of blaming the analyst-messenger.
Will the government be tempted to cut? The ONS is biting its nails over the damage the Comprehensive Spending Review might do to Whitehall’s analytic capacities, and ominous noises are heard in some departments about the fate of social research. ‘This isn’t a government interested in evaluation,’ says a senior civil servant. ‘The government is just not commissioning assessments of its policy initiatives.’
Maude thinks pilots are fine in cockpits, but is not keen on paying for them in advance of rolling out prized Tory policies. His track record is mixed. The UK Statistics Authority had to send him a polite but firm letter of reprimand for publishing figures on civil service employment based on a ring round of departments by his special advisers. The ONS itself had been about to publish up-to-date and certified numbers.
Two glaring examples of evidence-less policy-making stand out from the early months. Gove ploughed ahead with parent-run schools without pausing for breath; Health Secretary Andrew Lansley produced his GP consortiums plan fully cooked from his back pocket. Both might have been given geographically delimited trials and both appear to be examples of the hasty policy-making for which the Opposition caned Labour a few years ago.
The government also ceased funding speed cameras without a review or appraisal of costs and benefits. Culling quangos might cut independent appraisal; ministers have seemed unconcerned. The National Policing Improvement Agency – its abolition announced in the summer – has been working on the effectiveness of CCTV. Now there is a subject armchair auditors might have strong views on. What if they clashed with the objective evidence put forth by the analysts?
The government has taken against polling and opinion research. Communities minister Grant Shapps declared the biennial Place Survey a ‘burden’ on councils. This was the postal survey of residents views conducted by councils over the past couple of years against a common template of questions, with the results put in league table form.
Wasteful, said Shapps. ‘These surveys are a cosmetic exercise which never change anything. The idea that council bureaucrats are forced to turn themselves into amateur pollsters to ask highly intrusive personal questions about their residents seems entirely out of place.’ Yet councils had emblazoned Place Survey results on their websites, presumably as an attested and objective statement of residents’ views.
It would be premature to conclude the coalition has ended ‘evidence for policy’, the slogan on which Labour came to power in 1997. Remember how, for a time, former prime minister Tony Blair and colleagues were so keen on evidence that they barely let projects such as Sure Start and the New Deal for Communities get going before asking academics to pull them up by the roots and examine the results.
But Labour was none too keen on evidence getting in the way of ministers’ fixed beliefs. Examples are previous justice secretary Jack Straw’s conviction that prison worked and that more police officers cut the rate of crime. Dispassionate analysis suggested Home Office action accounted for about a fifth of the reduction in crime on Labour’s watch, the rest was caused by better employment prospects, training and college courses for young people and, most banal of all, a drop in the number of young men in the age groups most likely to commit offences. And, in a pamphlet out in June, former education secretary Estelle Morris ruefully admitted ‘pressure for quick fixes can outweigh research when ministers set schools policy’.
This summer’s judgement has to be that the government values evidence up to a point, and evidence of a quality that the armchairs could never amass. Despite blaring headlines in the Daily Mail last week, ministers chose not to second-guess the decision by the National Institute for Health and Clinical Excellence not to endorse Avastin, a drug used in late-stage bowel cancer. The press quoted Barbara Moss from Worcester complaining that Nice had put a value on life and saying she was ‘living proof’ that Avastin worked.
It was a classic instance of mistaking individual for average outcomes. The Guardian columnist Ben Goldacre has made fun for a long time of statistical folly and ignorance on the part of public, press and ministers. And gingerly – because no-one likes to criticise democracy – voices have begun asking about the potential of the armchair auditors. As well as drugs that are not cost-effective, they might favour homeopathy, for example. This is still provided on the NHS although the present government, like its predecessor, says it believes the clinical evidence that the treatments do not work. But – as the NHS Choices Website says – ‘there are many considerations when making policy decisions, and patient choice is an important factor to consider’.
In the Truth is out there, published in March, the Audit Commission wondered: ‘What information will capture the public’s imagination and therefore be used? Putting large volumes of data online will not by itself be enough to enable and encourage people to exercise choice or influence public decision-making. With widespread concern about the trustworthiness of public information, what assurance will members of the public need in order to trust the information made available to them?’
So much depends on the quality of the data eyeballed by the internet mavens. Who is to assess the accuracy of what the armchairs behold? The government website data.gov.uk wants councils to use similar templates for the information they put out, which ought to make comparison easier.
A government committed to evidence might, in theory, have researched the prospect for armchair auditors and other dimensions of the Big Society before they became policies. What do we know about people’s enthusiasm and capacity? Ben Page of Ipsos Mori says his surveys imply a ‘seismic shift’ would be necessary to get the involvement the government envisages.
Gillian Fawcett, head of public sector at the Association of Chartered Certified Accountants, says: ‘The reality is that very few members of the public currently look at local authorities’ accounts – even though that opportunity is available to them. In many cases, where people are interested in accounts at all, their interest will be restricted to a specific issue, often to an area of personal interest.’
But so far the audit professionals are relaxed about the changes. CIPFA’s Ian Carruthers talks of mixing and matching the best of the old regime and the ‘potential benefits of the new framework’. Still, a nagging question remains: who might audit the armchairs, and subject their views to scrutiny? Pollster Bobby Duffy notes how people rely on the media in forming their perceptions and sometimes do not let data get in the way; they regularly misperceive reality, for example, on knife crime or teenage pregnancy. ‘The public often get it badly wrong, but few dare say so.’
David Walker is director of communications at the Audit Commission; these are his own views. His book with Polly Toynbee, The verdict: did Labour change Britain?, will be published by Granta in October
For full coverage of the abolition of the Audit Commission, click here
