Performance anxiety, by Colin Talbot and Carole Johnson

11 Jan 07
Public Service Agreements were meant to keep track of what departments do with taxpayers' money. The problem is that no-one, least of all MPs, pays them much attention. Colin Talbot and Carole Johnson report on moves to reform a 'scrutiny lite' system

12 January 2007

Public Service Agreements were meant to keep track of what departments do with taxpayers' money. The problem is that no-one, least of all MPs, pays them much attention. Colin Talbot and Carole Johnson report on moves to reform a 'scrutiny lite' system

When the government introduced its flagship modernisation policy of Public Service Agreements in 1998, it was jumping on to a public management reform bandwagon. Governing by performance – and especially by outcomes – had become de rigueur.

It was part of an international fashion. One of the earliest and biggest examples of the approach was the Government Performance and Results Act, passed by the US Congress in 1993. Of course, like all fashions, the reforms have taken on their own local colour, spreading well beyond the usual 'Anglo' suspects – the UK, the US, Canada, Australia and New Zealand – to France, Japan and beyond.

According to David Osborne, the American public management guru who has heavily influenced New Labour thinking, government by outcomes has not been confined to federal government in the US but has swept across local and state authorities too. In the UK, PSAs were seen as a way of reassuring voters that extra government spending would be matched by reform. Chancellor Gordon Brown made it clear at the time that additional funding would be available only to public services that met their targets.

Almost ten years on, this outcome-based approach – and in particular PSAs – is coming under growing criticism. A Treasury consultation on the future of the agreements is doing the rounds in Whitehall. Meanwhile, the Treasury select committee is scrutinising PSAs as part of a wider inquiry into 'emerging issues' for the 2007 Comprehensive Spending Review.

Just before Christmas, a National Audit Office report raised serious issues about the quality of data underpinning the PSA and other target-based regimes. One in five of the data systems used by departments was found to be 'not fit for monitoring progress on the key elements of their PSA targets or… had not been established at all'.

NAO head Sir John Bourn warned: 'Without good data, monitoring against targets becomes highly devalued. If we are to have confidence in the performance reported by government against its key objectives, it is crucial that the data systems used to monitor it are robust.'

The problems with PSAs, though, go even deeper than the quality of their data. Governing by outcomes or performance has two main aims: improving management in government and improving the accountability of government. Most attention has been focused on the first – do the new systems really improve the economy, efficiency and effectiveness of governments? Much less attention has been paid to whether the new systems have improved accountability – especially accountability of the executive branch of government to the legislative branch. In short, is anybody listening to the findings?

This is especially important as in two of the four main 'governing by outcomes' countries (the US and France), the initiative for the new systems came originally from the legislative branch of government. And in the UK the government announced when launching them in 1998 that PSAs were to create 'a fundamental change in the accountability of government to Parliament and the people'.

So has it led to such a transformation? Has Parliament, in particular, adapted to the new PSA system? With four rounds of PSAs behind us (1998, 2000, 2002 and 2004), now is a good time to ask just what has altered in the way Parliament holds the government to account?

Members and officials of parliamentary select committees don't seem convinced that much has changed. A survey by Manchester Business School reveals deep scepticism about PSAs. One Labour backbencher said: 'I simply do not believe PSAs have had the slightest impact on public service providers or on users. They appear as a management gimmick of neither use nor ornament, but do cost money better spent on services. Weighing the pig never made it any fatter!'

The survey spent some time looking at how far select committees had actually used the PSA data made available to them. We looked at seven spending department select committees (defence, education, health, home affairs, work and pensions, the former office of the deputy prime minister and environment, food and rural affairs) and at their reports over a three-year period (2002 to 2005) to see how much attention they had paid to PSAs (see box on page 19).

The results certainly do not look like a 'fundamental' shift in accountability. Of 270 PSA targets for which data was available to these committees over the three years, we found that only 47 – less than one in five – were mentioned at all in the committees' annual report. This is despite the fact that Parliament's Liaison Committee, the joint body for all select committees, recommends regular scrutiny of PSAs in its set of 'core tasks' for committees.

The picture was slightly better in other committee outputs – 36 other reports referred to PSAs, out of 156 (23%). But detailed examination showed that more often than not the mention was superficial. One or two committees have produced more substantial reports (for example, home affairs), but overall the picture is one of 'scrutiny lite' when it comes to PSAs.

We also looked at the three committees that have some responsibility for scrutinising PSA policy: Treasury, public accounts and public administration. The latter had conducted a major review of all performance reporting in their On target report in July 2003. The Treasury committee had held hearings covering PSAs each time the new Spending Review and PSAs were published. Interestingly, the Public Accounts Committee, the senior committee for scrutinising the executive, appeared to have done very little at all on PSAs.

Why is this the case, when government supposedly sets such great store by PSAs and Parliament has masses of data available to it? We tried to find out from members of the select committees.

Asked if PSAs really were 'an important instrument of government policy', both MPs and officials seemed equally sceptical, only marginally agreeing that they were. Nor did MPs think PSAs had significantly changed the way government accounted for itself to Parliament.

On the quality of PSA data submitted to Parliament, both MPs and clerks were more or less neutral about whether the information was accessible, understandable, reliable, accurate and 'fit for purpose'. Both thought it was insufficient to judge performance.

An MP on the foreign affairs committee criticised the 'one-size-fits all' approach to trying to make all PSA targets focus on 'outcomes'. Targets drawn up in terms of inputs/outputs would be more relevant and more useful, one said. Another respondent from the education committee questioned the whole way PSAs were arrived at: 'Why have a target for 50% participation in higher education? The Department for Education and Skills was unable to offer a convincing explanation. Are the targets for achievements at Key Stages Two, Three and Four reasonable or appropriate? It has proved very difficult to have any sort of useful dialogue on these issues with the department.'

MPs, and clerks even more so, thought that PSA data were not widely regarded outside of Parliament as an accurate and fair assessment of government performance. MPs felt the government had not encouraged active scrutiny by Parliament. The clerks were marginally more positive.

Both groups were critical of their own role, with MPs feeling they had not been successful in scrutinising PSA data. One MP said: 'PSAs are one tool – but not the only way in which select committees can hold the executive to account.' Although there are exceptions – such as a home affairs select committee report on Home Office targets – in general our research shows that the level of scrutiny of PSAs by select committees is very low.

Both MPs and clerks thought they should report regularly on PSAs, but were not overly enthusiastic – and were slightly divided over whether committees should focus on specific issues rather than regular scrutiny of PSAs.

One MP said: 'PSAs rarely come up in our select committee evidence sessions with user groups, permanent secretaries and ministers.' Another told us: 'To the best of my recollection PSAs have never been mentioned in the select committee in the whole of the past 12 months.' A third added that PSAs were 'virtually never mentioned in the committees I am on'.

Neither group believed that they had sufficient resources to scrutinise PSAs properly. When asked whether the NAO should provide Parliament with an annual assessment of PSAs, they both supported the idea. One clerk suggested this would allow committees to 'draw down' information where there is cause for concern. Another emphasised that there have been occasions when PSA targets have affected accountability, such as Estelle Morris's resignation as education secretary after failing to reach literacy and numeracy targets. More frequently though, ministerial and public service performance are measured by other standards.

Overall, the survey results suggest that scepticism about the PSAs themselves, among MPs and officials, is feeding a lack of parliamentary enthusiasm for scrutinising them. No wonder then that little has been done to rectify the situation highlighted by the NAO, where so many of the PSA monitoring systems used across government departments are barely adequate.

The NAO recommends that the Treasury should 'challenge departments' measurement arrangements early in the process of developing new PSA measures and targets', and take other measures to strengthen data systems. However, given the importance that

the government attaches to PSAs, it is clear that more far-reaching steps will have to be taken.

The level of scepticism on the part of MPs and officials has not gone unnoticed in the Treasury, and a fundamental rethink of how PSAs are used is under way. It should be embedded into the next round of new, slimmed-down PSA targets due out with the CSR.

There are suggestions of a significant change of approach, away from the so-called 'outcomes' focus towards a much more tightly managed emphasis on outputs and efficiency, with integrated financial and non-financial management systems. Outcomes would not be abandoned, but dealt with through more long-term evaluation and policy reviews, rather than focusing on targets.

Ironically, such changes would come as an unwelcome shock to many spending departments. After initial resistance they have adapted well to PSAs, mainly because the long-term, outcome-focused targets offer huge scope for 'ducking and diving' around issues. The failure of current systems to be linked to finances in any obvious way makes this even more pliable. One senior official in Whitehall told us that the Treasury had concluded deals with several departments for CSR07, months before negotiations about PSA targets had even started. 'It's a funny old contract where you agree the price before you even discuss what you're buying,' as they put it.

If the rumours are true, these contracts might be about to get a lot more hard-edged – and governing by outcomes will become much more about governing by outputs. If they do, it will be interesting to see if Parliament also starts to take PSAs – or son-of-PSAs – more seriously as a tool for holding government to account.

Colin Talbot and Carole Johnson are, respectively, professor and lecturer in public policy and management at Manchester Business School

PFjan2007

Did you enjoy this article?

AddToAny

Top