Wednesday, May 21, 2014

Nice try, George Eaton, but it's time to reacquaint you with referendum polling reality

Well, we might as well all pack up and go home, chaps.  An oracle walks among us, and he has decreed that the best the Yes campaign can hope for is a narrow defeat leading to Devo Max.  And the name of this new rival to Nostradamus?  Er, well actually it's George Eaton, the tediously tribal Labour hack who also declared two months out from the 2011 Holyrood election that his party were coasting to victory, and that Iain "the Snarl" Gray was about to become our First Minister.  Result?  SNP landslide.

Someone asked me earlier today if I could respond to Eaton's latest fairy-tale reimagining of the opinion polls.  It turns out that comments on his New Statesman piece are already closed (after one day!) so I'll do it here instead.

"Labour MPs might be increasingly anxious about their party's performance in the national polls (today's Populus survey has them a point behind the Tories) but their spirits have been lifted by the latest numbers on Scottish independence. After narrowing for months, the polls have begun to move in the No campaign's favour."

Nope. "The polls" have not done any such thing. With the sole exception of the ICM poll, every single poll that has been published recently has either shown movement to the Yes campaign (extremely big movement in the case of Progressive Scottish Opinion), no movement at all, or movement to No that is so tiny that it is most likely to be margin of error 'noise'. As for the ICM poll itself, that could easily be one of the more extreme examples of 'noise' - if the true No lead according to ICM's methodology is roughly 7% or 8% (and two polls from the firm so far this year have shown exactly that sort of lead), then it would be quite possible for the standard margin of error to produce a 3% lead one month and a 12% lead the next without there having been any actual movement in opinion. And that's before we even mention ICM's bizarre methodological change which we only found out about more than 24 hours after the poll was published, and which means that it's impossible to make a direct comparison with previous polls in the series. To put it in a nutshell - without corroboration from at least one other pollster, the sensible assumption is that the trend shown by ICM is not real. Two pollsters have already had the opportunity to back ICM up - Panelbase and Survation both conducted polls that partially overlapped with ICM's fieldwork, and neither showed statistically significant change. Other pollsters will have the same opportunity in the days to come, but as of this moment there is literally no convincing evidence of any movement back to No whatever.

"while another by Panelbase (the Yes campaign's pollster of choice) put it [the No lead] up from five points to seven."

Which is precisely one of the statistically insignificant changes I referred to earlier, but as it happens even that is exaggerated. As I pointed out yesterday, if you look at the unrounded Panelbase numbers the No lead has actually only increased by 0.8%, from 5.7% to 6.5%. I don't know if there's a word for something that is even more insignificant than an utterly insignificant thing, but if there is, that word very neatly describes the statistical irrelevance of the increase in the No lead that Panelbase have just reported.

"Across the six main pollsters, the No campaign's average lead now stands at 14 points."

Actually, on a point of pedantry, it stands at 13 points. Eaton helpfully shows his working in inflating the number - he links to a tweet from "Jackanory" Jim Murphy that once again peddles the brazen lie that the recent Progressive poll showing a 9 point drop in the No lead was not in fact a Progressive poll at all, but was instead a YouGov poll showing a 6 point increase in the No lead. Just to reiterate the truth for the umpteenth time - the most recent polls from BOTH Progressive and YouGov show a DECREASE in the No lead, not an increase. YouGov's last poll was for Channel 4 News, and it showed the No lead narrowing from 15 points to 14 - a new low for the campaign. Progressive's poll was for the Sunday Mail, and it showed the No lead plummeting from 29 points to 20 - again, easily a new low for the campaign from the firm that used to be by far the most No-friendly pollster.

As for Eaton's unspoken implication that the average No lead has just dramatically solidified, in fact it has only recovered to a position that is 2.1% higher than its all-time low, which it sunk to just a few weeks ago. That very limited recovery is almost entirely due to the changes in the ICM poll, which as already stated must be regarded with suspicion - a) because no other pollster has replicated the trend, and b) because of the big alteration in methodology. In other words, the balance of probability is that No are in fact still stuck at their lowest level of support so far, and that Yes have successfully consolidated the gains they made during the winter.

"A narrow defeat might allow the SNP to press for devo max (and even to revisit the independence question at some point) but a defeat it will be."

Hmmm. Do you think we should ask First Minister Gray for a second opinion before we abandon hope completely?

*  *  *

I haven't been able to track it down again (I may be looking on the wrong thread) but there was a comment on Wings last night from someone who claimed to have it "on very good authority" that the unweighted results of the UK government's notorious secret Ipsos-Mori mega-poll were Yes 36%, No 48%.  If the weighted figures are close to that (and if anything Ipsos-Mori's weightings have tended to boost Yes slightly), then it would mean that the No lead of 25 points in the last published Ipsos-Mori poll has more than halved. Obviously there's no way of telling if the claim is true, but it certainly has the ring of plausibility to it - the rumours that Yes were in the lead never seemed very likely given Ipsos-Mori's track record as the most No-friendly BPC pollster, but this kind of whopping reduction in the lead would be entirely consistent with Westminster's determination to keep the poll under wraps.

Such a big turnaround from this campaign's only telephone pollster would mean one thing and one thing only : Game On.


  1. And the whole premise of Eaton's article is that a No result in the referendum will automatically help Labour.

    An alternative view could be that Labour have done so much during the referendum campaign to alienate much for their constituency in Scotland (many of whom would have voted Yes)that any comeback in Scotland would be delayed for many years.

  2. Sunshine on CrieffMay 21, 2014 at 9:29 AM

    Steve B

    You describe me. I have voted Labour since getting the vote way back in 1977, but this referendum campaign has opened my eyes. I will not vote for them at any level until they change their attitude towards Scotland. And rid themselves of their aggressive and narrow-minded British nationalist element.