Wednesday, December 24, 2014

Christmas Poll of Polls shows SNP leading by 18%

Remember how I said last night that "unless any newspaper is crazy enough to publish a poll on Christmas Eve, that really should be that"?  Well, it turns out that the Daily Mirror is crazy enough, and they've just published a GB-wide Survation poll.  We've had an unexpected little flurry of three GB-wide polls over the last 48 hours, so for the sake of completeness I'm going to do another Poll of Polls update - I'm sure this will be the last one of 2014 (although I seem to have said that somewhere before).  It's based on one full-scale Scottish poll from Survation, plus six Scottish subsamples from GB-wide polls - three from YouGov, two from Populus and one from Survation.

Scottish voting intentions for the May 2015 UK general election :

SNP 44.1% (-1.7)
Labour 26.5% (+1.2)
Conservatives 16.5% (n/c)
Liberal Democrats 5.8% (+0.5)
UKIP 4.0% (+0.3)
Greens 1.7% (-0.2)

(The Poll of Polls uses the Scottish subsamples from all GB-wide polls that have been conducted entirely within the last seven days and for which datasets have been provided, and also all full-scale Scottish polls that have been conducted at least partly within the last seven days. Full-scale polls are given ten times the weighting of subsamples.)

The final Scottish subsample before Christmas proved to be the oddest of the lot, with Survation putting Labour well ahead of the SNP (by 48.9% to 27.1%, to be exact).  That's only the second non-Populus subsample since the independence referendum not to have the SNP in the lead.  However, there are three very good reasons why that finding is probably not significant -

1) It's based on a particularly small sample of 69 people, prior to weighting.  For comparison, a YouGov subsample will typically be based on interviews with more than 200 respondents.

2) It's not bang-up-to-date.  It may be the last poll to be published before Christmas, but it wasn't the last one to be conducted.  The more recent subsamples from Populus and YouGov have continued to show the more familiar pattern of a huge SNP lead.

3) Survation's fieldwork even partly overlapped with their own full-scale Scottish poll, which showed a mammoth 24% SNP lead.

There's also the fact that Survation weight their results by recalled Westminster vote, which we know harms the SNP.  But that's something all firms other than Ipsos-Mori do in their GB-wide polls by one means or another (if they don't literally weight by recalled Westminster vote, they weight by Westminster-centric party identification figures).

The bottom line is that it's almost certainly just a freakish result, but it's one more drop of evidence to be put into the mix with all the other drops, and the overall picture we're left with is the SNP lead of 17.6% that you see in the Poll of Polls.

All the same, this is a timely reminder of how the huge divergence between Scottish and English voting intentions may be making GB polls less reliable.  Labour have a 3-point lead across Britain in the Survation poll - but if we assume the Scottish subsample is just a freak caused by normal sampling variation, that lead may not be particularly meaningful.

*  *  *

A small technical note : I realised tonight that I've been looking at the wrong table in previous GB-wide Survation datasets - I had assumed that Survation use the numbers which have been adjusted for the "spiral of silence" in their headline results, but it appears that they actually use the unadjusted numbers instead.  That means there may have been slight inaccuracies in a small number of previous Poll of Polls updates.  However, the inaccuracies will have been very slight indeed, so I'm not going to torment myself by trying to retrospectively correct them.

*  *  *

I know we're all reeling with disbelief after the terrible events in Queen Street and George Square, but I'd still like to wish all readers of this blog a very Happy Christmas.


  1. The SNP lead seems to be getting smaller and smaller :/ it better not slump back down to pre referendum or the Unionists are gonna be all over it

    1. "Smaller and smaller"? What?! The last update showed an SNP lead of 20.5% - that's easily the highest it's been for weeks. It's now slipped to 17.6%, which is still higher than it's been most of the time since the referendum.

      The long-term trend is upwards, John, not downwards.

  2. The sub sample partially overlaps the full scale Survation poll, and we have two YouGovs and a Populus conducted on the same dates or after, all showing huge SNP leads. It's a freak result.

    As for the trend, it's pretty obvious that full scale Scottish polls give a truer picture of where things stand than sub samples, and the trend is an increase in the SNP lead.

  3. Actually Survation don't weight their GB results by recalled Westminster vote (neither do Opinium by the way); they did at one time but then stopped. They do weight their Scottish polls by recalled 2011 vote and now by recalled referendum vote as well - though confusingly they don't say this in their methodology section. As it happens this Survation poll recalled vote (40/30/23/8) matched actual (37/30/24/10) pretty closely after other weighting.

    One thing that is odd about this though is that the Scottish sample is so small. Normally Scots tend to be pretty enthusiastic about replying anyway and that has been particularly true this yea with the higher level of political engagement . But in this poll the 74 had to be weighted up to 86 - compare for example today's YouGov where 216 respondents from Scotland had to be down-weighted to 143.

    This may well have something to do with the two surveys being so close, though if they're using a big enough panel that shouldn't matte3r.

    1. According to the methodological note at the top of their datasets, Survation do weight by target figures from the 2010 general election. I suppose it's possible that they might have forgotten to correct out-of-date information, but a cursory glance at the numbers certainly gives the impression that they weight by past vote recall.