Saturday, September 13, 2014

Yes campaign open up eight-point lead in stunning poll from "gold standard" ICM

Just one day after their extraordinary telephone poll that put Yes within 1% of victory, the UK's "gold standard" polling organisation ICM have gone one better by producing an online poll showing Yes in an outright lead - and a lead of eight points, no less.  The percentage changes listed below are measured from the last ICM online poll, because that's the one it's directly comparable to,

Should Scotland be an independent country?

Yes 54% (+9)
No 46% (-9)

There's also a new Opinium poll out tonight - no percentage change figures are possible for this one, because it's the first referendum poll Opinium have produced.

Yes 47.4%
No 52.6%

Since I began this post, a Panelbase poll has also been published, so I'm going to analyse all of today's four polls in a fresh post.  Before I do that, though, I just want to deal with a point about the ICM poll that has already been discussed at length in the comment section below and elsewhere.  It is categorically not the case that it should be regarded as a lesser poll because it had a slightly lower sample size than usual (705 as opposed to the more typical 1000).  According to the very handy MOE calculator on the ComRes website, the margin of error for a poll of 700 respondents is approximately 3.7% - which is only fractionally higher than the 3% margin of error for a poll of 1000.  Someone suggested below that I have said in the past that, by definition, polls of this size cannot be regarded as statistically reliable.  I've never said that, because it isn't true.  The only thing I can think of is that I put a health warning on YouGov "polls" of roughly this size in the run-up to the European elections, but that's because they weren't proper polls - they were aggregates of subsamples from GB-wide polls.  The key factor in determining whether a poll is legitimate is not the sample size (unless it is absurdly small) but rather whether it was properly weighted.  As far as I know, this one was weighted in exactly the same way as any other ICM online poll would be - I'm quite sure Professor Curtice would have been the first to flag it up if that had not been the case.  So it should be regarded as a legitimate ICM online poll in exactly the same way as any other ICM online poll.

Indeed, we've had smaller polls than this during the campaign - two of the three Angus Reid polls that were published last year had samples of 500 or so, and although that fact was noted in passing, it wasn't perceived as significantly detracting from the credibility of the results.  In the US, samples of 500 would be regarded as fairly routine, and it would certainly be seen as very odd to use that as an excuse for ignoring any given poll.  And what about the ICM poll with a sample of 500 which the media seized upon as 'absolute proof' that Alistair Darling had defeated Alex Salmond in the first debate?  I don't recall Euan McColm casting doubts on the reliability of that one.  (I did, as it happens, as did Professor Curtice to some extent, but that was mostly because of the extreme upweighting that had gone on.)

It's therefore extremely troubling that the title Curtice uses for his blogpost on the ICM poll is "ICM Put Yes Ahead - Perhaps".  Does the "perhaps" refer to the 3.7% margin of error?  If so, shouldn't any verdict on any standard poll of 1000 people also automatically have the word "perhaps" attached to the title, because all of those polls have margins of error of 3%?  Or is there some kind of mystical gulf between the numbers 3 and 3.7 that I've failed to comprehend so far?  It just seems like a ludicrous double-standard.  I note that a report on the BBC website is mentioning all of the other polls today, but not the ICM one - I can't help wondering if that's been caused by a misinterpretation of Curtice's title as meaning that this is somehow not a "proper poll".  By the way, I have no complaint about Curtice's decision to add this poll to his Poll of Polls, but to give it a slightly lower weighting in proportion with the sample size - that seems perfectly reasonable to me.  But it should also be seen absolutely, unambiguously as an endorsement of the poll's legitimacy, because that's exactly what it is.

You probably don't need me to point out that you would have to go to the extreme end of a 3.7% margin of error to take this poll to a result that does not have Yes clearly in the lead.  Given the strength of the evidence from other pollsters that No are either slightly ahead or level, perhaps we do need to assume that ICM have produced a result that is at the extreme end of the margin of error, but so what?  We could be forgiven for making exactly the same assumption about today's Survation poll, which is showing a lower Yes vote than others.

Further analysis of all of today's four polls, plus a Poll of Polls update, will appear HERE.

105 comments:

  1. Still need to fight for every vote. They can still steal it, don't let them.

    ReplyDelete
  2. There will be a marked absence of Trolls.

    Curtice says only 750 polled so must be treated with caution.

    Fuck that!

    http://blog.whatscotlandthinks.org/2014/09/icm-put-yes-ahead-perhaps/

    ReplyDelete
  3. Holy fuck.

    But...

    It's just a poll.

    No complacency. No fear. Fight to the end. And we will win. Yes all the way.

    ReplyDelete
  4. Are we counting this one James? As it's a smaller number polled...hope so!

    ReplyDelete
  5. opinimum for observer has 53N/47Y but is it just me or hav they not included 16-18 year olds in the poll?

    Data tables here
    https://twitter.com/ShippersUnbound/status/510770109447942144

    ReplyDelete
  6. Are we counting this one James? As it's a smaller number polled...hope so!

    ReplyDelete
  7. This should count, margin of error up to 3.8% instead of 3.2% due to smaller sampling.

    ReplyDelete
  8. Laukat ....Opinium suspiciously late on the scene.

    Not counted on James's poll of polls.

    Can't describe how I felt when I saw this Poll on Twitter.

    ReplyDelete
  9. Think Curtice has a point unfortunately...James has warned about using lower numbers polls before.

    ReplyDelete
  10. I don't want there to be a big lead for us, the establishment have already been in full scale panic mode and it hasn't worked. what will they try next?

    ReplyDelete
  11. We should ignore Opinium and go with ICM...nearer the truth.

    Chris D

    ReplyDelete
  12. Not that much lower and an eight point lead. come on!

    ReplyDelete
  13. "I don't want there to be a big lead for us, the establishment have already been in full scale panic mode and it hasn't worked"

    Popcorn time.

    ReplyDelete
  14. Dave - thanks its method does look a little odd

    ReplyDelete
  15. Hilarious! The Court of King James goes into overdrive.

    ReplyDelete
  16. Full tables: http://news.opinium.co.uk/sites/news.opinium.co.uk/files/observer_scotland_final.pdf

    "For UK general election polls we make our samples match published ONS figures for gender, age group, region, working status and socio-economic grade. Our weighting factors for Scotland were a variation of this with gender, age group, working status and socio-economic grade being the same while region was calculated by asking respondents which county they lived in and combining these into larger regions to be used in weighting."
    http://news.opinium.co.uk/opinium-blog/explanation-our-scottish-poll-methodology

    I presume they haven't waited for country of birth. If other online pollsters anything to go by they could have too many England-born people.

    As with other pollsters, far too few people did not vote in 2011.

    Age Yes % look reasonable.

    But only 26% of Lab are Yes.

    ReplyDelete
  17. Dr Éoin Clarke ‏@LabourEoin
    Opinium's Poll found an innovative way to measure momentum. Yes vote growing & hardening. No vote floundering

    ReplyDelete
  18. Maybe not much lower in numbers but enough to question the statistical validity required for sampling...Think that's the point James has pointed out before. Believe me, I'd like to rely on it but the scientist in me says I can't. :/

    ReplyDelete
  19. Now we know why BT released their internal polling. You can't ignore this ICM poll.

    ReplyDelete
  20. Now we know why BT released their internal polling. You can't ignore this ICM poll.

    ReplyDelete
  21. If you use the standard formula to work out sample size for a 5 million population then that would tell you this sample should produce around a 3.8% margin of error by my reckoning.

    However this isn't actually an exact science. You also have to keep in mind that the margin of error has a 95% confidence interval (i.e. you'll get within that margin of error 95% of the time, the other 5% of polls will be outside it).

    So it could be something genuine, or it could be nonsense. The polling is so all over the place at this point that I'm not sure anyone really knows what's going to happen on Thursday now. Curtice gets a lot of stick for being cautious and saying things like "we need to see another poll show this before we can be sure there's a trend" but when you realise the limitations in polling I don't blame him for being that way (and those are the limitations when it actually works how it's supposed to, far less when it's flawed).

    ReplyDelete
  22. @Aldo
    Actually the polling is currently much more stable than it's ever been in the campaign. This latest ICM is an exception.

    ReplyDelete
  23. @keaton

    You don't buy this ICM then?

    ReplyDelete
  24. I'd take it over a poll that showed the opposite result, obviously, but if we're to assume that polls have any worth at all I can't see any reason to give much weight to one that's about ten points out from the general consensus right now.

    ReplyDelete
  25. Guess the Panelbase will help us see if the lower sample ICM is an outlier? Think it will be similar to ICM 54+ yes so Murdoch can launch Sun for yes.

    Chris D

    ReplyDelete
  26. @ at Keaton

    Even though it confirms what canvassers are finding on the doorstep?

    ReplyDelete
  27. 400+ of us turned out to a rally in support of a YES vote in Cardiff today. In contrast 7 turned up for Dan Snow's rally in Cardiff last month.

    For more info click my name above. You guys are an inspiration to us down here.

    Saor Alba.

    ReplyDelete
  28. @David

    Agree...but why aren't the others doing the same or the ICM couple of days ago...I fear an outlier. Any comments yet James?

    Alan

    ReplyDelete
  29. Scotland is in party mood.

    Our MSM don't want to report this as it does not fit their agenda of project fear.

    ReplyDelete
  30. Aye better together clearly 'at it' with that survation poll,they knew this was coming or they were worried about panelbase. Very interesting from icm.

    ReplyDelete
  31. ICM certainly does back up canvassing returns.

    ReplyDelete
  32. @Chris
    There's no evidence that Panelbase will have anything like that figure.

    Sometimes I think we're still going to be telling each other next year that the Sun is about to declare for Yes.

    @David
    No, not really. People are more likely to be honest with ostensibly neutral pollsters than canvassers who wear their affiliation on their sleeve.

    A lead for Yes of that size sounds plausible here in Glasgow, but not with Edinburgh and the borders in the mix.

    ReplyDelete
  33. My feeling is that we are in a state of extreme fluidity right now - I think people are shifting to YES almost by the hour. So a lot depends on the sampling. In a few days time the fog might clear...or might not....exciting stuff! :D

    ReplyDelete
  34. @keaton

    Not long until I can say "told you so" ;)

    Chris D

    ReplyDelete
  35. This might be a daft question, but why did ICM commission a poll with only 750 respondents?

    And if there is 750 respondents, why can't we use the 'weighting' we hear about from all the polls in order to make this 750 respondents into virtual people, ie add maybe 75 virtual people to each grouping to make it more than 1000?

    Or can James or Scottish Skier take the data and increase each grouping to create enough people to make it 1000.

    eg if 50 respondents were between 25 & 40 years old and from social group D and 24 said Yes, 17 No, 9 DK, we then add 15 virtual people using the same voting percentages as the real people.

    Or am I missing something?

    ReplyDelete
  36. @Chris
    I hope you get to do so!

    ReplyDelete
  37. I think Edinburgh will be No overall, but not massively so - perhaps 53% to 47% :-) Glasgow and surrounds most likely Yes probably by the reverse. Dundee is apparently Yes. What about the north? What's the story there?

    ReplyDelete
  38. @keaton

    I have heard that there are a lot of Yes signs in Marchmont windows.

    ReplyDelete


  39. I'm not too sure about that - as S_S never fails to remind us, some people tell porkies to pollsters too.

    ReplyDelete
  40. oops! Don't know what happened there - but the above was an answer to keaton's post about people being honest with pollsters.

    ReplyDelete
  41. Icm may chime with skiers theory regarding shy yes....less than a week to go,hours after the most brutal assault on us from the union and people now saying, fuck it. You are getting it.

    ReplyDelete
  42. Are we expecting a panelbase poll as well tonight?

    ReplyDelete
  43. chalks - is this announcement or something you were talking about a couple of days ago still on the cards?

    ReplyDelete
  44. Agree with Keaton but crikey what a night. Right now the polls are 2-1 for No but Panelbase might equalize. A hell of a lot for James to get his teeth into.

    ReplyDelete
  45. Backing up the point earlier, Opinium have overlooked the 16 to 18 years.....is that right, how does that affect things..?

    ReplyDelete
  46. proff Curtice saying that the ICM poll is legitimate! he doesn't seem to have the doubts expressed on here.

    ReplyDelete
  47. Opinium did poll 16+ voters ...tweeted it...error on table label

    ReplyDelete
  48. Curtice says "with a substantial health warning"...didn't dismiss it out of hand though

    ReplyDelete
  49. Panelbase:

    Yes: 49.4%
    No: 50.6%

    Will be 51-49 after rounding.

    ReplyDelete
  50. Momentum still with YES after PB and ICM polls.

    ReplyDelete
  51. Wtf Panelbase...bunch of bent vipers...49 no way...

    Bastards

    Enough for Murdoch to jump though.

    :/

    Chris D

    ReplyDelete
  52. The two ICMs plus PB suggest momentum is back with Yes. Interesting.

    ReplyDelete
  53. Hmm makes the ICM a bit of an outlier? Was a bit worried on the lower numbers...WHY didn't they poll 1000 then we'd know?

    Alan

    ReplyDelete
  54. We can discount the internal poll as an attempt at deflection. The Opinium poll has no provenance. The ICM and PB polls show a move towards YES.

    ReplyDelete
  55. Woof!

    49.4 to 50.6 ! Isn't that exactly Quebec's numbers?

    Way to go.

    ReplyDelete
  56. Stating the obvious...it's a strange world where we wait on that evil Australian sod to help our cause. OTOH...if he helps us get our country back, we'll be able to show him the door politely but firmly thereafter.

    ReplyDelete
  57. Thought Quebec were 57 before polling day? I'm probably wrong

    ReplyDelete
  58. I was polled by ICM (Newvista, which James thinks is ICM), quite late on (yesterday evening) so it could be that they asked the required number of people but Newvista left the email too late for people to answer in time?

    it was also quite a long poll, with some bizarre questions mixed in. Some people might have got fed up and closed the browser (for example, at one point they asked you to do a reaction test, clicking on red dots as they appear on the screen)

    ReplyDelete
  59. Murdoch's papers will clinch it for us now...don't care what he does afterwards

    ReplyDelete
  60. I'm certainly not relying on Murdoch. An endorsement from him could well do more harm than good. Fortunately I don't expect one to be forthcoming.

    ReplyDelete
  61. We would need an IMMENSELY long spoon to sup with that one..

    ReplyDelete
  62. Robert, West LothianSeptember 13, 2014 at 9:47 PM

    Think we can forget that earlier BT poll designed only to grab headlines and deter Yes voters. They knew later polls tonight would show a much closer contest. Keep fighting guys and girls we can do this.

    ReplyDelete
  63. The polls still aren't showing what canvassers are finding on the doorsteps. All to play for, but I'm growing in confidence. Westminster is heading towards a perfect storm of 'Yes' voters.
    I'm looking for 'Yes' to be ahead by a good margin heading to the polls. Monday morning financial markets will be interesting. What fresh shit can Westminster unload on us without appearing to be the panicking twats they actually are. Will there be a CU concession by Gideon? If there is we've won.

    ReplyDelete
  64. Kellner predicting no...part

    Chris D

    ReplyDelete
  65. @David
    As you say, if he agreed to a currency union Osborne would likely hand victory to Yes. So there's zero chance of him doing so.

    Don't know why people keep dreaming up these fantastic scenarios.

    ReplyDelete
  66. BVB was told a prominent businessman would be in the FT endorsing yes....didnt quite materialise and with the markets going mental I think he wasnt comfortable going the whole hog....

    This will be enough for murdoch.

    ReplyDelete
  67. Pardon? Who cares what he predicts? Is he mystic Meg now?

    ReplyDelete
  68. Meant prat not part!!!

    ReplyDelete
  69. Thanks Chalks. Shame.

    ReplyDelete
  70. @Keaton

    If the markets panic what is he going to do?

    ReplyDelete
  71. I think Scotland might actually do a "reverse Quebec".

    ReplyDelete
  72. @chalked

    You think Murdoch won't go Yes?

    Alan

    ReplyDelete
  73. Did wonder what "Part" meant. Sadly he's pretty experienced. Where and when was this prediction? He might have changed his mind already!!

    ReplyDelete
  74. @David
    Welcome it as ammunition against Yes. Same as he did last time.

    ReplyDelete
  75. Kellner in Sunday Times apparently

    Alan

    ReplyDelete
  76. 'he's pretty experienced'

    An old Labour whore would be a better description.

    ReplyDelete
  77. Regarding the Observer/Opinium poll, who was doing indy polls for the Observer previously - assuming there are older Observer indy polls? I was trying to search their website, but parsing out Guardian stuff is difficult.

    I think they use ICM for UK party political polls - maybe they have done ICM polls they're not comfortable with, and not published? Not very credible to turn to a company who haven't done indy polls at all, just a week before the vote!

    ReplyDelete
  78. So four polls tonight, 3 show a swing to Yes, one significant but also covers the period the other "pro-No" polls swung to Yes. So that is consistent. One poll, requested by Westminster shows a very marginal swing to No on a different methodology.

    I really think the only reason No published the private poll was because without it, we would have two ties and one yes, all of which were moves to Yes.

    And again, NOT ONE POLL counts the 200,000 newly registered voters who are not First Time voters which can only be predominantly Yes.

    ReplyDelete
  79. I think they will pull the trigger on it on monday Alan.

    Apparently the sun editor in caledonia is chomping at the bit. Just london that are hesitant. But seeing as how murdoch was all over scotland today, he is building up to it.

    Not sure if it will add lots more, but its very welcome to have pro indy headlines and stories.

    ReplyDelete
  80. Murdoch went to his ancestor's home. He isn't retarded, he knows why they left. They left because the Union forced them to leave. Sentiment might actually work in our favour.

    ReplyDelete
  81. We are seeing a swing to YES over the past few days when BT, their big business cronies and the BBC have thrown the kitchen sink at the Yes campaign.

    BT have nothing left.

    ReplyDelete
  82. The BBC has not been as uniformly partisan as some here think...for instance, Robert Peston's repeated scepticism on the "Prophets of Doom" from the banks and businesses. Perhaps the YES campaign should point to his observations more?

    ReplyDelete
  83. BY are far from dead yet...don't write them off! We have to keep working as hard as ever!

    ReplyDelete
  84. Preston has gone a bit jungly recently...sometimes he sounds pro-yes but then you reread it and he isn't

    Alan

    ReplyDelete
  85. Tomorrow's Sunday Telegraph headline: Scottish soldiers lost their lives trying to preserve the United Kingdom. What will their families say now: 'Well, it no longer matters'?

    Can't see that winning round many undecideds and soft Yeses.

    ReplyDelete
  86. Daily Telegraph' s best spin on today's polls "too close to call".

    They really are bricking it now.

    ReplyDelete
  87. ''Scottish soldiers lost their lives trying to preserve the United Kingdom. ''

    Or in other words, many Scottish soldiers were used as cannon fodder for Englands many wars of aggression.

    ReplyDelete
  88. With the polls so tight, I think anyone claiming victory for either side is simply publishing their own bias.

    I feel that the turnout of the electors will tip the balance, but even that has its own questions. I would expect the BT will win the postal vote due to the majority of postal voters being older than 55. The interesting result of the postal vote will be the margin of victory.

    Conversely, I think that Yes Scotland will win the vote at the polling stations due to the overall strength of their ground game and the absence of a large amount of over 55’s having already voted by post.

    In the end, If BT does not have a sizable lead in postal votes, at least 10-15 percent in my view, then YES will carry the day.

    Nothing is guaranteed either way at the present time and more polls are to come, but this is how I see at the moment.

    ReplyDelete
  89. Last Night of the Pwoms is on BBC1 right now, I had the misfortune to see 30 seconds of it, all the Hooray Henry Twits waving their Union Jacks.

    Quite nauseating, practically vomit inducing.

    ReplyDelete
  90. British journalism today and then they wonder why more and more people are voting Yes.

    ReplyDelete
  91. I know no undecided voters actually buy the Sunday Telegraph, but that piece of insanity is going to be sitting at newstands tomorrow. Must be worth a wee mini-boost for Yes.

    ReplyDelete
  92. ''Murdoch went to his ancestor's home. he knows why they left. They left because the Union forced them to leave. Sentiment might actually work in our favour.''

    Murdoch is Australian, like Scotland, they've always been looked down upon and considered outsiders, never accepted by the snobbish London Elites. Murdoch has a brooding resentment about it.

    ReplyDelete
  93. '' I know no undecided voters actually buy the Telegraph.''

    You should read the gibberish from tory twit Daniel Hannan gushing about the glories of the Bwitish Empire. He must think it's still 1914.

    ReplyDelete
  94. still dribbling on about london, tories, the english etc... this is why you're going to lose in the end, you started out naming groups of people they didn't want the votes of, smart move :-)

    Murdoch is teasing, with the establishments in the uk, us etc against independence, he'd be hanging his media empire out to the wind, he'll hint, but I doubt he'll move :-)

    Salmond will not divide the uk, but he will divide scotland :-(

    104 hours till polls open...



    ReplyDelete
  95. ''still dribbling on ''

    Yes, you seem to be.

    ReplyDelete
  96. amazing comeback there david brims of froth :-)

    ReplyDelete
  97. Sunday Telegraph Headline '' Scottish soldiers died to defend the Bwitsh Union.''

    I thought the First World War was about defending Belgium's Independence. The Second World War was about defending Poland's Independence.

    As for the Iraq war, Saddam wanted to sell his oil in Euro's instead of the U.S Dollar thus undermining the American currency, so the Federal Reserve invaded. Libya wanted to do the same.

    So Scots soldiers died for other nations Independence, bankers and oil corporations, not to defend the Bwitish Union.

    ReplyDelete
  98. I used to enjoy the Last Night of the Proms. A bit of fun to end the summer after singing one's way through about eight weeks of it. Did I really not notice, or has the jingoism been ratcheted up over the years? (I last sang in the choir there in the mid 1990s.)

    I was leaving the pub this evening with our Plaid volunteers who had been helping with the leafleting, and the car radio came on at Radio 3 automatically (best station to avoid referendum propaganda) and then I realised what was on. Smart shift to Radio 4 and frankly listening to someone saying it was too close to call was a relief.

    ReplyDelete
  99. Perhaps you unionists should take a look at what Cameron will say to the Scots next. He is dividing the UK, the North will turn on him in a massive way.

    ReplyDelete
  100. I think this weekend's polls are sitting at exactly where I'd like them to be going in to the last few days of the campaign. Most are showing a statistical dead-heat, one is veering towards a No and another is pointing to a Yes, but with a question mark hovering over it because of the smaller sample size .... leaving enough doubt so as to not spook the financial markets, and enough hope to give Yes supporters a huge shot in the arm. My psychic powers tell me that the Yes campaign will clinch it by the slenderest of margins, smaller even than that obtained by the Nos in Québec. I am thinking < 4,000 votes.


    ReplyDelete
  101. If so, I know to whom I give the credit for gathering in those vital 4000 votes. And he's nothing to do with Yes Scotland and he doesn't even live here. #WBB

    ReplyDelete
  102. I drive a blackcab ,edinburgh,even NO voters are saying they understand why YES voters are voting YES


    Noticed a change about a week ago,a huge change from a year ago.


    If better together kept their mouth shut from the beginning,the union was probably saved,now I think vote is a close YES

    ReplyDelete
  103. Just thought you'd all like to know that the demonstration down at the BBC made the ABC TV news here in Australia, with the commentator putting the attendance figure at over a thousand. Certainly looked very impressive

    ReplyDelete