Tuesday, January 28, 2014

ICM poll looks robust

Many thanks to Stuart Dickson for pointing me in the direction of the full datasets for the remarkable poll that has, for the time being at least, turned the independence referendum campaign on its head. Last night I expressed a small amount of concern that the SNP's lead on European voting intentions in the poll seemed improbably high, which I thought might cast some slight doubt on the credibility of the referendum findings. However, I'm delighted to report that the datasets have entirely set my mind to rest on that point. The only real question mark anyone had raised about the poll's methodology related to the upscaling of the small number of 16-24 year old respondents, among whom there was a huge swing towards Yes. But as Oldnat pointed out on last night's thread, if a disproportionately 'Nat-heavy' sample of young people was responsible for the Yes surge, you would expect that to be reflected in huge support among that group for the SNP. As it turns out, the opposite is the case - 16-24 year olds may be the part of the poll's sample most likely to vote for independence, but they're also the least likely to vote SNP...

SNP support by age group -

16-24 year olds : 36%
25-44 year olds : 47%
45-64 year olds : 44%
65+ year olds : 38%

So there's nothing suspicious at all about the SNP's handsome lead, which in fact is derived from the responses of older age groups for which there were no sampling problems. It's admittedly still a very surprising finding, but my theory is that the order of questions in the poll may have had something to do with it (European voting intention was only the ninth question asked, and was immediately preceded by a question asking people to recollect how they voted in the 2011 Holyrood election).

As for the referendum voting intentions of young people in this poll, the truth is that no-one should be startled by them - at 44% Yes, 33% No they're in line with the finding from the most recent YouGov poll that 18-24 year olds were breaking disproportionately for Yes. If anything, a more reasonable concern might be that the traditional sampling problems among young voters led to an underestimation of the Yes vote in the last ICM poll in September (the Yes figure for 16-24 year olds in that poll was absurdly low). If that was the case, then it might cast doubt on just how big the swing to Yes has been over the last four months, but what it wouldn't do is cast any particular doubt on the headline figures that ICM are showing right now - ie. a No lead that stands at just 7%.

What really seems to be going on is that voters of all ages are gradually learning to decouple their party political preferences from their referendum voting intention. The Yes campaign still don't seem to be making much headway among Tories (if Wealthy Nation do the trick on that front it could yet prove decisive), but among Labour and Liberal Democrat voters the progress is extraordinary. When undecideds are excluded, no fewer than 28% of people who voted Labour in 2011 are planning to vote for independence, as are 26% of people who voted Liberal Democrat. And remember that both of those parties were at historically low levels of support in 2011, so these are the true core voters who are being won over by the Yes campaign. By contrast, in spite of the fact that the SNP were way, way above their core support when they secured their 2011 landslide, a mere 16% of their voters are currently planning to vote No.

In no small measure, it may well be that we have the likes of Sir Charles Gray and Dennis Canavan to thank for this breakthrough. We can only imagine what the impact will be if a few more senior Labour people are now emboldened to declare for Yes.

* * *

Would it be possible for journalists (and Blair McDougall) to stop inventing their own poll numbers?

Blair McDougall discussing the ICM poll on Scotland Tonight : "Every poll has shown Yes at between 25% and 33%." Even if we're charitable and assume the implied missing words were "until this one", he's still talking utter tripe. No fewer than ELEVEN polls last year showed the Yes campaign at a higher level of support than 33% - and that includes two polls from Ipsos-Mori, generally the least favourable pollster for Yes. The highest Yes figure of all was in fact 44%.

Ian Dunt (yes, him again) writing on politics.co.uk : "The ICM poll is something of an outlier, however. Most surveys put support for independence at around 29%." WHAT? Where in God's name does that number come from? Even when undecideds are not excluded, the average Yes vote at present is 33.8%. Five of the six BPC pollsters showed a Yes vote of over 30% in their most recent poll (and even if the non-BPC Progressive Scottish Opinion is taken into account, it's five out of seven). Has Dunt just plucked the number 29 out of thin air because he likes the sound of it? Answers on a postcard...

Last but not least, although I didn't see Question Time last week, I was told on Twitter that David Dimbleby lied through his teeth yet again by claiming polls were showing a 2-1 majority against independence, and that (almost as unforgivably) none of the panel bothered to correct him. This just ain't on - not even Dimblebys have the right to unilaterally decide that undecided voters are really No voters in denial. Not only should he immediately stop peddling this falsehood, he should apologise for having grossly misled his viewers for so long. And even if he is genuinely unaware that he's been reciting fictional numbers, the fact that he didn't even bother to check speaks to an almost unbelievable lack of professionalism on his part.


  1. I suspect you'll only see Senior Labour figures break rank if it appears a YES win is on the cards.

    Once it starts though it will be a rush for the exits.

    At that point they'll be thinking about personal career options rather than toeing the line. At this point standing up for YES as a current Labour man is a real gamble.

    I think the Tories already have their darling waiting in the wings in the shape of Murdo Fraser.

    Maybe even a chance for wee Charlie Kennedy to make a Libdem comeback. Danny boy will however be consigned to the scrapheap to join such greats as Comical Ali in the event of a YES

    Other illustrious names that will be casualties will be Darling, Brown, Forsyth, Foulkes and Davidson(x2)...Sadly I suspect Lamentable will survive as walking wounded and will continue as MSP, as will Wee Wullie Winkie.

  2. We have to be careful with the polls and our reaction to them. It is nice to get some positive news for the YES campaign that just can't be ignored by Belter Toegether and the media but I often wonder how truly representative the polls are. The debate I feel would be better if we could just not give as much coverage to them as a tool for influence. The message is getting across slowly but surely for the YES Campaign but they have to keep the momentum going, ensure that complacency does not creep in. I would be happy not to see any polls for a while.


  3. I think that polls have a 'lag time' compared to talking to folk.

    I appreciate that everything I say here is just personal experience, but the movement towards 'YES' reflects my experience of talking to folk.

    Unscientific I know, but there certainly is movement. Perhaps the apocryphal reflects the reality?

    Is that so hard to believe?

  4. Its a bit of a mixed bag for me, I see lots of wavering and have a few folks in my circle who will never vote YES and refuse point blank to even discuss it meaningfully. You just get an enigmatic No way we couldn't on our own or Salmond is a ...So its not reasoned NO voting in my experience but ingrained negativity.

    They remind me of my old man who was Labour and would not even discuss Nationalism nor independence.

  5. We should expect a new Panelbase poll in the next few weeks, I've just taken one.

    I've taken part in their surveys before, and I thought you might be interested to know that they always use a preamble they don't show in their results tables:

    "As you may know, the Scottish government intends to hold a referendum this year on Scotland becoming a country independent from the rest of the United Kingdom. The question on the ballot is expected to be as below. How would you vote in this referendum?

    Should Scotland be an independent country?"

  6. Hi Calum, that really is appalling, in two ways - firstly, because the wording is tortured and potentially leading, and secondly, because they don't own up to it.

    Are you sure that they've always done it? Because if (in the nightmare scenario) this is a new wording it might affect the results quite significantly. We know what happened when YouGov finally dropped the leading words from their own preamble last September.

  7. This is maybe my 4th survey in indepdence over the last year, and I am 100% that they have always used that preamble.

    The wording of the question isn't that big of a surprise considering that the Sunday Times commissioned it, but the fact you would never have known unless you had taken the survey is strange.

    I'd say the exact same about Ipsos as well. They mention in their summaries they ask respondents how they would vote tomorrow (I remember a man from Ipsos even saying that on STV news), but no sign if a preamble has ever been in their results tables. Judging by the no leads they manage to produce, there's a chance that it is very leading indeed.