Monday, July 14, 2014

ICM poll is better for Yes on the unrounded numbers

It's uncanny how often this has happened with ICM - to be fair it's just coincidence/bad luck, but the rounding has yet again flattered No in the published results of the new poll for Scotland on Sunday.  On the headline figures that take account of Don't Knows, the true No lead before rounding is much closer to 10% than to the 11% we saw in the published numbers.  And when Don't Knows are excluded, Yes are at 43.44%, and are therefore just the tiniest smidgeon away from being rounded up to 44% rather than down to 43%.  Here are what the poll results look like when rounded only to one decimal place -

Should Scotland be an independent country?

Don't Knows excluded :

Yes 43.4% (-1.8)
No 56.6% (+1.8)

Don't Knows not excluded :

Yes 34.3% (-1.5)
No 44.6% (+1.2)

So the changes from last month (which the barking mad Huffington Post described as a Yes "nosedive") are even smaller than they appeared to be in the published results, which increases the overwhelming likelihood that they're just meaningless margin of error 'noise'.

You might also remember that after revealing the results of the ICM poll from two months ago, Kenny Farquharson made a big issue of the fact that if the firm's methodology hadn't just changed to take account of differential turnout, then Yes would have slipped to 40% (actually 40.4% on the unrounded numbers) rather than to 42.4%.  Well, If you scour your memory of the last 36 hours or so, you'll probably realise that no-one has made that point this time, and for a very good reason - in this case the Yes vote before turnout weighting is applied is the same (43%) as in the published results.  So whichever way you look at it, the No lead is clearly lower in this poll than it was two months ago - and by quite a substantial amount on one measure.

I'm slightly baffled as to how the Scotland on Sunday and John Curtice felt able to make a song and dance about how this poll supposedly shows that people are starting to trust the anti-independence parties on the delivery of new powers to the Scottish Parliament in the event of a No vote.  In reality, the number of people who think no new powers will be delivered or that existing powers will be taken away is, at 45%, higher than the number of people who expect new powers to arrive (40%).  It's possible that the electorate may eventually have the wool pulled over their eyes by the No campaign, but it certainly hasn't happened yet.

Now let's have a look at what John Curtice didn't tell us about the question asking people to rate themselves on a 1-10 scale of support for independence.  The first thing to say is that this only turns up deep, deep into the question sequence, which is less than satisfactory, because by that point many respondents will be self-consciously trying to keep their responses consistent with what they've already said.  Nevertheless, the results are much better for Yes than we've been led to believe.  As a number of people confirmed on the previous thread, it's standard during canvassing to regard a '5' as being exactly in the middle (ie. a complete Don't Know), meaning that anything between 1 and 4 is on the anti-independence end of the scale, and anything between 6 and 10 is on the pro-independence end.  Pedants and statisticians may bristle at that notion, because it means there are slightly more pro-independence numbers available, but nevertheless that's how it works - the human brain is programmed to see 5 as the mid-point in a scale that goes up to 10.  When looked at that way, these are the results -

Pro-independence (6-10) : 43.5%
Anti-independence (1-4) : 48.9%
Neutral (5) : 7.6%

(The above numbers exclude the small number of respondents who refused to place themselves on the scale.)

Now are you starting to see why the Yes campaign are so bullish about the numbers that are being produced by that approach?  The other point Stephen Noon made the other day (and unfortunately the ICM poll can't assist us with this one way or the other) is that, over time, people have been gradually moving up the scale towards independence, and that was happening even before any movement towards Yes was detectable in the published polls.  If that's true, it could be hugely significant.

Of course it also means that the people who currently rate themselves as a 5 are particularly important, as they literally hold the balance.  So what do they look like?  Well, 40% of them didn't even vote in the 2011 election, so that limits the numbers of clues available.  22% voted Labour, and 12.5% for the SNP.  Most say they are undecided on how they will vote in the referendum, but a significant minority of 24% currently say they will vote No, which goes a long way towards explaining the disparity with the headline numbers - these are the soft Nos we keep hearing about.

There's also a weird amount of "cross-voting" going on - supposed No voters who place themselves on the pro-independence end of the scale, and supposed Yes voters who place themselves on the anti-independence end of the scale.  Were these people just confused by the question?  It's likely that at least some of them were, given that 1% of No voters rate themselves as a 10, ie. "completely for an independent Scotland"!  But the 5% of No voters who rate themselves as a 6, 7, 8 or 9 are potentially more significant - those could well be soft Nos (very, very soft Nos).  By contrast, just 1% of Yes voters rate themselves as a 2, 3 or 4.

Annoyingly, ICM haven't asked a country of birth question this time (or if they have there's no sign of the results).  But on past form it seems reasonable to assume that English-born people are once again heavily over-represented in the sample, which almost certainly means that the headline Yes vote is a bit too low, and the headline No vote is a bit too high.  However, women aren't quite as over-represented as they were in the previous two polls, which is a touch irritating given that (very unusually) more women than men in this poll say they will vote Yes!


  1. I don't want static/margin of error. I want sharp, decisive moves to Yes!

  2. You say you were irritated by some missing information. Does that mean you think they could have been tampering with the selection to fit the end result?

  3. No, the issue with country of birth isn't a suspicion of tampering - it's that ICM are failing to adjust their results to take account of the fact that they've got too many English-born people in their panel. Being born in England is still a strong predictor of a No vote, so this error will be distorting the headline numbers to some extent.

  4. Tories take the lead in ICM for the Guardian.

  5. Total newb at this hoping someone can help, looking at the ICM data tables for the recent Scotsman poll.

    It appears the largest single No voting group are people who didn't vote at the 2011 Holyrood election.

    226 people who did not vote were polled and are weighted up to 315 people. Of this weighted total 134 are voting No.

    A quick calculation of DNV and voters shows that this group makes up 34% of the entire No vote.

    I'm hoping someone with better understand of polling can confirm/or correct the previous?

    It seems to me that up weighting the largest no voting group by nearly a 1/3 would make the No percentages more prone to error?

    I would presume also that the DNV category are selected to fit the economic and regional demographics given there is no indication of their political opinions based on who they voted for at an election? Or are they selected in some other fashion?

    Please forgive any glaring errors above I just happened to glance at the tables and became curious with no prior background in such analysis.

  6. Good analysis James though I'm still struggling to see why some were desperately pretending that one poll spelled 'certain doom' for the Yes campaign.

    On the ground polls like that are hardly taken very seriously since they are so out of kilter with what Yes campaigners are hearing on the doorsteps and in the streets.

    Just saw that poll Scottish_Skier. The tory twits will be in amusing full hysteria mode but that's a poll that will send shivers up the spines of Labour and 'No Better Thanks Together'. (or whatever they are called this week.)

    Interesting to note that's a big kipper drop. The fact is sooner or later the kippers were obviously going to drop just like they did last year.

    The tories still aren't taking full advantage of that seven point drop since their rise is only three points. However, with the incompetent fop Cammie blundering around cluelesly as usual that's hardly a surprise either. His latest wheeze appears to be to boot out Ken Clarke to please gullible tory eurosceptics after his second flounce. It will likely work as well since tory eurosceptics are notoriously gullible and want to be fooled.

    I also can't see the Blairites keeping quiet for much longer with little Ed struggling so badly. I've said it before and I'll say it again, any halfway competent Labour leader would be annihilating someone as scandal prone and lightweight as Cameron is. It's absolute proof little Ed is one of the worst Labour leaders we've seen. Little Ed is being handed ammunition on a plate on an almost daily basis (some of which could topple a PM it's so incredible) yet Little Ed seems more concerned with arguing among his spads and shadow cabinet over just how much he can triangulate on the tories. Pathetic.

    Speaking of pathetic, Calamity Clegg is charging his ostrich faction and what remains of the yellow tories over the cliff with ever greater speed. With the rebranded snoopers charter debacle Clegg seems more determined than ever to push the lib dems towards complete fringe party status and irrelevance.

    The upshot of all this is that the scottish public are being shown just how untrustworthy and beneath contempt the current crop of westminster parties and leaders are. Sooner or later that crucial "trust factor" is going to kick in to the referendum with a vengeance. I'm going to enjoy it hugely when it does because the response from the No campaign will be a blind panic that will overshadow all that has come before.

  7. @ Mick Pork

    My own analysis has Labour consistently below 35% for the past few months now. That means zero chance of a majority and likely to cause Ed to move ever more right under Blairite pressure.

  8. I wouldn't argue with that Scottish_Skier though I would simply add that these past few months have been among the most benign you could possibly imagine for a Labour leader.

    There's been a seemingly never ending stream of scandals from Cammie and the tories, there's the kippers still splitting the right wing vote AND the toxic calamity Clegg has managed to do the impossible and push the lib dems DOWN from a seemingly unshiftable 10% they had been on since late 2010.

    If in 2010 you told labour MPs they were getting all that handed to them on a plate, a year before the election, their response would have been ecstatic. Yet despite all those massive advantages little Ed and labour are, as you so rightly point out, basically flatlining on a dire low score for them with little Ed about as appealing to the public as Clegg. (which is as bad as it gets)

    There was a big song and dance about the Blairites having been defeated when a couple of the more prominent ones moved on to more lucrative undertakings. The point that misses is that the Blairites don't need any figureheads or standard-bearers now. They simply push little Ed further and further to the right by questioning his leadership along with many disgruntled labour MPs.

    With little Ed under that much pressure it's hardly a surprise he's too weak to stand up to them and keeps appeasing the Blue Labour hardliners.

  9. Big twit reshuffle going on.

    Hammond as Foreign Secretary. LOL

    Hague quits.. curious timing indeed. Lots of rumours that he is going before scandal forces his hand. We shall find out in due course.

    Lansley booted out on his arse. Poor old PB tory twits, always wrong.

    Dominic Grieve is made the fall guy for the fop's bizarre and idiotic forays into law that almost resulted in the hacking trial being stoped.

    The incompetent fop is trying to deal with his woman problem again by plonking in a few tokens. Cammie's not an inept PR man for nothing.

  10. Anon at 7.05 : Sorry that your comment disappeared - for some reason it got caught by the spam trap. I'll try to answer your questions later.