Battle of the pollsters

When I first heard whispers a couple of hours ago that there was a new poll out showing a substantial No lead, I can honestly say the first thought that went through my mind was "I just hope to goodness it's a YouGov poll". Reasons? Well, there were three main ones -

1) It would mean that No have nothing much to celebrate, as there has only been a very modest increase in their lead since the last comparable poll (which, remarkably, was published almost a year ago!).

2) YouGov are notorious for using a leading preamble when posing the referendum question, and can probably be assumed to have done the same thing this time, thus detracting somewhat from the credibility of the results (although I'll happily apologise tomorrow or on Monday when the full tables go up, if it turns out that for the very first time they have let the official Electoral Commission-approved question speak for itself).

3) There are increasing question marks over YouGov's Scottish sampling. A lot of water has passed under the bridge since the 2003 Holyrood election, when YouGov towered over their old-school competitors as by far the most accurate pollster of the campaign. They also performed reasonably well in 2007, but by 2011 the tables had well and truly turned, and YouGov were now the 'establishment' pollster being outshone by their less prestigious rivals. They continued to show a tight race right up to polling day, entirely failing to pick up the impending SNP landslide. The problem seems to have got even worse since then - the Scottish subsamples in most YouGov daily polls this year have shown the SNP trailing Labour badly, and in many cases even languishing in a thoroughly implausible third place behind the Tories. By contrast, the subsamples of all other pollsters have typically shown the SNP either in the lead, or only very slightly behind Labour. It doesn't automatically follow that YouGov are getting it wrong - we have to at least entertain the possibility that they are getting it right and everyone else is getting it wrong. But intuitively it seems much more likely that there is an ever-growing rift between real-world Scotland and YouGov's Scottish samples.

Thankfully, this weekend's poll did indeed turn out to be a YouGov poll. That means we don't learn anything new, but we do have a reaffirmation of the gulf between the two pollsters that are showing a relatively modest No lead (Angus Reid and Panelbase), and the three pollsters that are showing a more substantial one (YouGov, TNS-BMRB and Ipsos-Mori) - although YouGov find themselves at the extreme end of the latter group. So which group is getting it right? Or is the truth somewhere in between the two? You pays your money and you takes your choice, although it's worth pointing out that media commentators who tell you that "No have a handsome lead, full stop" are failing to take account of the complete picture - probably willfully.

Curiously, this YouGov poll was commissioned by the Devoplus campaign, with the apparent intention of producing results that would demonstrate to the No camp that they would have a better chance of victory if they committed to increased powers for the Scottish Parliament. That doesn't seem to have really come off - the Devoplus crowd would probably have been better advised to use a pollster that doesn't routinely produce inflated No leads in the headline figures.

By the way, have you noticed that when we see a poll showing a modest increase in the No vote, there are headlines screaming about "the No vote hardening up", but when Angus Reid showed a two-point increase in the Yes vote a few days ago, it was billed as a "no change" poll? I'm confused...