If you thought that the days were over when the chances of predicting our political future hinged on judging the merits of different polling methodologies, think again. Just look at the huge difference between the results of two simultaneous Britain-wide ComRes polls which have just been released - the first of which "prompts" for UKIP (ie. it reminds respondents of the party's existence in exactly the same way that it reminds them of the existence of the other 'main' parties), and the second of which does not.
Britain-wide voting intentions for the May 2015 general election (UKIP prompted for) :
Liberal Democrats 7%
Britain-wide voting intentions for the May 2015 general election (UKIP not prompted for) :
Liberal Democrats 7%
Incredibly, the bizarre methodology used for the second poll is the "norm" for ComRes, as indeed it is for many other pollsters, and this calls into question whether even the unprecedented numbers UKIP have been enjoying recently may be a significant underestimate of their true support. If they really are in the mid-20s and just seven points behind the first-placed party as the "fairer" poll above suggests, then we are well and truly into an "all bets are off" scenario, because there are still plenty of opportunities for Farage to build further momentum before the general election - most obviously there's the Rochester and Strood by-election next month, but there's also the lingering possibility of further defections from both the Tories and Labour.
Earlier this evening, I had a brief Twitter exchange on the "prompting" issue with Keiran Pedley of NOP, which used to be a big name in UK voting intention polling, although they haven't been active in recent times.
Keiran Pedley : @Nigel_Farage thinks UKIP should be prompted but where is the evidence that prompting UKIP gives a more accurate poll rating?
Me : To be fair, there seems very little chance that it would give a less accurate rating.
Keiran Pedley : we don't know yet - thats the reality
Me : Is there any reason why it wouldn't be a sensible precaution to prompt for UKIP?
Keiran Pedley : historically only prompt on 'main' parties, thing to remember is pollsters job not to be 'fair' but to be 'right'
Keiran Pedley : but given uniqueness of situation - v difficult to judge whats right approach re UKIP. Big challenge for pollsters
Me : But if you prompt for the fourth most popular party but not the third...surely that's bonkers?
Keiran Pedley : probably - get where you are coming from but then what if pollsters overstate UKIP as did LDs in 2010...its tricky
Keiran Pedley : thing about polling is it is literally both art and science
The comment about the "uniqueness of the situation" will perhaps remind you of the problem we faced before the independence referendum - in spite of the complacent boasts that certain pollsters (naming no names, but Peter Kellner) were making about the accuracy of their results, we knew that they were all fumbling around in the dark to some extent because they didn't have any previous independence referendum to "work backwards from" to ensure that their methodology worked in practice as well as in theory, ie. the "art not science" part of the process. Hence the gulf between the Yes-friendly and the No-friendly pollsters, and we're now seeing a similar gulf between polls that prompt for UKIP and polls that don't.
On reflection, I would concede the point that it is theoretically perfectly possible that the surveys which do not prompt for UKIP will turn out to be the most accurate, but if polling firms are depending for their accuracy on seemingly random decisions about which parties they remind respondents about the existence of, then I'm not sure that merits the billing of either "art" or "science". It's more like the method by which a broken clock occasionally manages to be more accurate than a clock that is consistently two minutes slow.
The reality is that when voters are faced with a ballot paper next May, they will be "prompted" for all parties, so pollsters should surely be doing their level best to replicate the voting experience as closely as possible. Perhaps the objection would be that voters are "frivolously" telling pollsters that they will vote UKIP, even though they will not actually do so when the government is being chosen. But there is simply no realistic way of controlling for that possibility, and pollsters shouldn't even try. Discouraging people from remembering that UKIP even exists seems like a particularly insane way of trying to control for it.