Whoever wins the Democratic nomination in the US, the opinion polls will be the losers. Primary after primary, poll-based newspaper headlines have announced that Barack Obama is in the lead or Hillary Clinton clear. Primary after primary, the ballot box has told a different story.
No doubt, pollsters will claim that they share the blame
with reporters. Indeed, reading the British press, it has been impossible to get any clear idea of what is actually going on in the US.
In particular, the purple prose of reporters’ accounts turns out to be the emperor’s new clothes hiding ignorance.
For example, few have told us which are ‘open’ primaries, in which any voter can choose to participate, and which ‘closed’, open to registered Democrats only.
You would not expect a poll of all voters in a state to predict a closed primary restricted to a small selection of them, any more than in Britain a poll of all electors would tell you what Labour voters think.
The US poll debacle raises an obvious question: could it happen here?
For in the US we have seen that rare creature, a large-scale trial of poll predictions against actual election results. The polls have been found wanting.
We have no primaries in Britain. So that leaves a fundamental problem with polling as a technique here. Only every four to five years, at a general election, is it possible to measure findings on voting intentions against the litmus test — the results of a real election.
Their record on this test in the past has been mixed. Sometimes they have been close to predicting the actual result. Often they have been quite close.
And at least twice in recent history they have been totally, disastrously, embarrassingly wrong: in 1970 and more recently in 1992, in both cases having Labour in front when the Conservatives romped to victory.
Various techniques have been introduced to try to avoid a further catastrophe — in particular, adjusting results to reflect past voting.
These techniques remain controversial and, besides, we wouldn’t have much faith in a scientist who fiddled his trial results before publication.
So can we be at all sure that recent polls are accurate? They put Labour four or five points behind the Tories, a deficit to be sure, though one that would not deliver the Conservatives an overall majority. But is it wise to put any faith in these figures at all?
There are plenty of good reasons for doubt. First, statistically, one in 20 polls will be a rogue, outside the standard margin of error of 2%—3% on each figure. This danger is, of course, reduced when a number of polls show much the same thing — there is safety in numbers.
Secondly, the margin of error itself is not insignificant. Take a smallish sample poll that puts Labour on 32%. This could in fact mean Labour is on 29% — a poor result. Of course it could mean it is on 35% — close to the vote that won it the 2005 general election.
But, thirdly, the margin of error is itself subject to a huge and unknown margin of error. For the statistical concept assumes that the pollster has a perfect sample — that is to say the 1,000 or so voters interviewed accurately reflect, or can be weighted to reflect, the views of the adult population as a whole.
This is getting harder and harder to achieve. On a conventional poll, it is not uncommon for half the voters approached to refuse to answer any questions. If these refusers are a cross-section of voters generally, that does not matter.
But what if they are not? What if they are (at one extreme) the most ignorant voters who don’t want to display their ignorance or (at the other) the best-informed voters who don’t think they should trust others with their private views?
As to which of these possibilities, if either, represents reality, we do not have a clue. Yet if either is the case, the sample is likely to be biased, in ways beyond the pollster’s, let alone the public’s, ability to comprehend.
Reporters don’t help either. Opinion polls remain expensive to commission, and newspapers expect a good story.
So you will read even respected commentators writing that Labour is up two points on last month and the Tories down one. This is usually followed by a more or less sophisticated explanation of why this chance has happened — Gordon Brown’s good speech, David Cameron’s gaffe, that sort of thing.
The real story — that statistically there has been no change whatsoever between the two months — has the advantage of being accurate, but that palls besides its disadvantage, that it is dull, dull, dull.
So is Labour really four or five points behind the Tories? If you must have an answer, wet your finger and stick it in the breeze. But if you want the right answer, it is that no-one has even the first clue.
