Sunday, September 1, 2013

Battle of the pollsters

When I first heard whispers a couple of hours ago that there was a new poll out showing a substantial No lead, I can honestly say the first thought that went through my mind was "I just hope to goodness it's a YouGov poll". Reasons? Well, there were three main ones -

1) It would mean that No have nothing much to celebrate, as there has only been a very modest increase in their lead since the last comparable poll (which, remarkably, was published almost a year ago!).

2) YouGov are notorious for using a leading preamble when posing the referendum question, and can probably be assumed to have done the same thing this time, thus detracting somewhat from the credibility of the results (although I'll happily apologise tomorrow or on Monday when the full tables go up, if it turns out that for the very first time they have let the official Electoral Commission-approved question speak for itself).

3) There are increasing question marks over YouGov's Scottish sampling. A lot of water has passed under the bridge since the 2003 Holyrood election, when YouGov towered over their old-school competitors as by far the most accurate pollster of the campaign. They also performed reasonably well in 2007, but by 2011 the tables had well and truly turned, and YouGov were now the 'establishment' pollster being outshone by their less prestigious rivals. They continued to show a tight race right up to polling day, entirely failing to pick up the impending SNP landslide. The problem seems to have got even worse since then - the Scottish subsamples in most YouGov daily polls this year have shown the SNP trailing Labour badly, and in many cases even languishing in a thoroughly implausible third place behind the Tories. By contrast, the subsamples of all other pollsters have typically shown the SNP either in the lead, or only very slightly behind Labour. It doesn't automatically follow that YouGov are getting it wrong - we have to at least entertain the possibility that they are getting it right and everyone else is getting it wrong. But intuitively it seems much more likely that there is an ever-growing rift between real-world Scotland and YouGov's Scottish samples.

Thankfully, this weekend's poll did indeed turn out to be a YouGov poll. That means we don't learn anything new, but we do have a reaffirmation of the gulf between the two pollsters that are showing a relatively modest No lead (Angus Reid and Panelbase), and the three pollsters that are showing a more substantial one (YouGov, TNS-BMRB and Ipsos-Mori) - although YouGov find themselves at the extreme end of the latter group. So which group is getting it right? Or is the truth somewhere in between the two? You pays your money and you takes your choice, although it's worth pointing out that media commentators who tell you that "No have a handsome lead, full stop" are failing to take account of the complete picture - probably willfully.

Curiously, this YouGov poll was commissioned by the Devoplus campaign, with the apparent intention of producing results that would demonstrate to the No camp that they would have a better chance of victory if they committed to increased powers for the Scottish Parliament. That doesn't seem to have really come off - the Devoplus crowd would probably have been better advised to use a pollster that doesn't routinely produce inflated No leads in the headline figures.

By the way, have you noticed that when we see a poll showing a modest increase in the No vote, there are headlines screaming about "the No vote hardening up", but when Angus Reid showed a two-point increase in the Yes vote a few days ago, it was billed as a "no change" poll? I'm confused...


  1. Baron Ashton of UphollandSeptember 1, 2013 at 10:07 AM

    Latest YouGov Scottish samples from UK polls.

    SNP lead over Labour:

    YouGov: -20%
    Populus: +2%
    ICM: -3%
    ComRes: -6%
    TNS-BMR: +1%

    SNP lead over Tories:

    YouGov: -5%
    Populus: +25%
    ICM: +28%
    ComRes: +20%
    TNS-BMRB: +23%

    Everyone's out of step except our Peter.

  2. The poll asked Yes voters, 'if the No camp offered greater power how would you vote?' However they did not ask a balancing question asking No voters 'if no further power are being offered how would you vote?'

    John Curtice on his blog implied that without that question the poll is undermined.

  3. The facts don't matter as far as the bigots in charge of the media ar concerned.

    They got their screaming headline and another round of undermining democracy. Well done to them.

    How long will the vote no campaign pay to keep their lapdog media alive after a no vote? Does anybody in the, "Scottish," media even care that they are working towards their own destruction?

    1. love it we in scotland will have our day and i hope to god we have a yes vote because scotland can and will run like the weather but we will come out the other side with the sun shining bright as we have the resources because if we didnt England would have never invaded years ago but they need us more than we need them thats why they do the scare mongering the same way they are trying to take over the world UK-USA coat of arms no more scotland wants its freedom and i want the freedom now

  4. Don't worry - you won't have to apologise re the preamble. I took the poll and it was there. What I can't remember is the order of the questions which might have had an impact on the response to the Yes/No question.