Thursday, September 11, 2014

ICM telephone poll poser

Thanks to Xabi on the previous thread for letting me know that ICM are conducting a referendum poll for the Guardian - and unlike the firm's series for Scotland on Sunday, this one is being done by telephone rather than online.  Believe it or not, this will be the first national telephone poll of the entire campaign to be conducted by a company other than Ipsos-Mori, who of course have tended to produce much more No-friendly results than the online pollsters, including ICM themselves.  It's unclear how much of this divergence is due to the telephone factor itself (and in particular the apparent landline-only nature of the polls), and how much is due to other distinctive features of Ipsos-Mori's methodology, such as failure to weight by past vote recall.

This development poses a problem for this blog's Poll of Polls, which uses the most recent poll from each firm, meaning that an ICM telephone poll will be replacing an ICM online poll in the sample.  That could well distort the trend, because there's a good chance that telephone polling will produce different results (my guess is that they'll be more No-friendly, but there's no way of knowing that for sure).  The alternative would be to treat "ICM (online)" and "ICM (telephone)" as two separate firms for the purposes of the Poll of Polls - but that might give too much weight to a single firm, particularly given that they'll presumably be using exactly the same methodology other than data collection.

Any thoughts?  For the sake of transparency, it would probably be best to make a firm decision well before we actually know what ICM are showing.

40 comments:

  1. You could create a one-off poll of polls marked "including ICM Telephone Poll", then any further polls could be included on the pre-existing poll of polls. Bit messy though :-S How annoying of them!

    ReplyDelete
  2. I think ICM polls are better together. ;)

    Given that other companies have also introduced relevant changes in their methodology throughout this long campaign, and that it has not led to any alteration of the Poll of Polls, I propose that you simply substitute the old ICM poll with a new one, with a warning, perhaps.

    Xabi

    ReplyDelete
  3. I'd have thought you can't replace an online poll with a telephone one because it's not consistent, so it's probably better to treat it as separate to their online method.

    ReplyDelete
  4. By the time it's published and dissected on this blog it'll be pretty much a moot point, it'll be referendum day

    ReplyDelete
  5. Anon : No, it won't be a moot point - the poll is being published a lot sooner than that.

    ReplyDelete
  6. Even if it's released soon and it's good for No (imagine this landline will be) then we'll have Panelbaseand YouGov to counter it.

    Is it not for the Hootsman too?

    ReplyDelete
  7. Hootsman ICM on Sunday, this is for guardian

    ReplyDelete
  8. Could they do a poll on whether the general public are still looking at polls in the papers.

    ReplyDelete
  9. Are we on course for a YouGov poll later? After all this financial misdirection I am in need of a little cheering up, and I think we might get it from this poll.



    ReplyDelete
  10. I was polled for the first time in my life on Sunday by Ipsos Mori.

    Not sure when that will be released or who it was for.

    ReplyDelete
  11. Would your methodology allow you to average the ICM polls out; either by finding an average result seperately and entering it into your model as ICM combined or by entering it twice but weighting ICM(Phone) and ICM (Online) to 0.5.

    This would make your model a little more complicated, but is a reasonable compromise and future proof because it will allow you to update the online portion as well, without an undesirable yo-yoing of your PoP.

    ReplyDelete
  12. TheMadMurph : I presume it was by landline?

    Anon : Yes, there'll be a YouGov poll tonight, and please don't get excited about it - I think we're way overdue for a reversion to the mean from YouGov.

    ReplyDelete
  13. I don't think a land line telephone poll could claim to reach any of the 'missing million' and will disproportionately represent those available to take a land line call (ie, at home). As its not part of a continuous poll by this method I don't think it can be trusted for trend either. My perception is that many parts will have to be up weighted so much that it'll distort the results. I see it showing NO much more strongly in the lead, perhaps as much as 60%/40%. Those commissioning the poll must know this though, mustn't they?

    ReplyDelete
  14. I can't see a landline poll being friendly for Yes, they'd have to seriously upweight in the younger dmeogrpahics from ridiculously small sample sizes.

    Also, with YouGov - if we expect a reverison to the mean then it's gonna be a rough couple of days for us in the press. Well ,relatively speaking - every day is a rough day for Yes in the press.

    ReplyDelete
  15. Cicero Elections ‏@CiceroElections 2m
    We are expecting a ComRes #IndyRef poll both tonight at 18:00 and another on Monday.

    ReplyDelete
  16. I'm wondering about the "big news" chalks said on a previous thread would be coming today.

    When can we expect this? Have we already had it?

    ReplyDelete
  17. Dalriadan : Hmmm. Well, that takes us up to seven pollsters. ComRes routinely do both online and phone polls, so I don't know what to expect. Probably online, because it's cheaper to commission.

    ReplyDelete
  18. I agree, folks can't see a landline poll being good. I was recently in a telephone call through work with 5 different people I've never met (all from Scotland). Someone asked how we were voting and 4 of us said undecided. I know for a fact, that 2 of my workmates - one has a Yes twibbon the other a Yes badge, are voting Yes - yet they said to this person they don't know that they were "not sure yet" and the other "No idea. I just want what's best for Scotland"

    How can you seriously get a honest answer from a landline poll? This is obviously a selected method by the Guardian to use a landline - to print more shite like "Blow for Salmond, Yes 38 No 49 etc" or whatever numbers they receive.

    I think the YouGov poll may be bad news too. Not terrible, but I think we'll see No retake the lead - not seen any Tweets or indications/hints/cyrptic messages from anyone, but I reckon it'll have swinged 3-5% back to No.

    Just need to hold tight. They are trying to take their wind out of our sails, but we have our bonus coming up - tonight's debate with the excellent Harvie and Sturgeon plus the joys of the Orange Order and UKIP campaigning in Scotland.

    Sit tight.

    ReplyDelete
  19. I think the ComRes poll is only for the south of Scotland. They've done that before. It won't be of much use for an overall picture of the country, but it could help give us a picture of the direction of travel when compared with their two previous polls in that region in January and June this year.

    ReplyDelete
  20. I think you should leave it out of the poll of polls until you have more than one of them.

    I mean if an unknown pollster (say from Europe or the US came and did a poll) how would you incorporate it?

    The alternate is to have two versions of the poll of polls one with, one without.

    ReplyDelete
  21. "I think the ComRes poll is only for the south of Scotland."

    Oh, God, not another of these pointless ITV Border polls. Why do they throw their money away on this rubbish? It's not even a poll of the south of Scotland - it's restricted to the Borders and Dumfries and Galloway, which accounts for just 5% of the population.

    ReplyDelete
  22. Wings over Scotland reported yesterday on Twitter that they had heard a "very surprising rumour" that would be "good" for the Yes campaign. They underlined that they could not tell what it is. This resonates with Chalks' big news he has promised not to reveal. Any announcement in coming days? Will it be enough to bridge the gap that apparently still exists between Yes and No? We will see.

    Xabi

    ReplyDelete
  23. Hi Xabi,

    My advice would be not to speculate about it too much - if it's sensitive information, then speculation could have a negative effect on whether it actually happens or not. In any case, the whole thing may have just been an unfounded rumour.

    ReplyDelete
  24. Perhaps Yes should commission a poll of Dundee. It would be equally worthless, but would at least provide some balance.

    ReplyDelete
  25. That's a reference to the Dumfries & Galloway-only ComRes, obviously, not ICM!

    ReplyDelete
  26. Just checked, and chalks didn't actually specify that this "big news" would be revealed today. Ho hum.

    ReplyDelete
  27. Hi, James,

    Perhaps it was not clear from my previous comment, but I was responding to Keaton.

    On the other hand, I believe that people in Wings over Scoland would not have revealed it on Twitter if they didn't want to create some hype about this.

    Still, you are right that this is mere speculation.

    Best,

    Xabi

    ReplyDelete
  28. Agree, James.

    We'll soon find out if this rumour that is apparently good for Yes is any use or not, but we need to get sucked in by the hype.

    Remember Severin Carrell I think it was who said Yes were sitting on a Panelbase lead for them, yet it showed nothing of that at all when it was released.

    And yesterday, we had some Yessers saying it was 53-47 to Yes when it was the opposite with Survation.

    All this "hearing good news from an SNP councillor" stuff does nothing, and ulimately leads to being downbeat if it's the opposite when the data is releaed. It doesn't help when you have blatant cheap points scoring from Survation describing a poll as "VERY interesting" and "quite something" when in fact it was nothing at all and exactly the same as their previous one 47-53.

    Keep on rolling folks.

    Think of the weekend!

    Fingers crossed for a great bounce tonight from Harvie and Sturgeon.

    ReplyDelete
  29. *we DON'T need to get sucked in by the hype*

    ReplyDelete
  30. @Kevin
    The No types frequently make up good news for Yes in order to manage expectations.

    Any Yesser who propagates positive rumours about an upcoming poll because they heard them from some random on Twitter wants locking up.

    ReplyDelete
  31. Sorry, Keaton!

    I mean't to say that, not Buster, hahaha.

    Apologies.

    ReplyDelete
  32. Update Sturgeon withdrew, Galloway back in. Read into that what you will but you really all should calm down a bit. You are sounding rather frantic.

    ReplyDelete
  33. Good afternoon, Mr Bravely Anonymous No Troll.

    ReplyDelete
  34. the border poll is to supply evidence for the possible upcoming partition of scotland. Areas that vote 60%+ no cant be expelled from the uk against their wishes after all. Self determimation and all that :-)

    ReplyDelete
  35. Hi there, Lord Kilclooney. Can you confirm that the inescapable logic of your argument is that, in the event of a No vote, the parts of Scotland that voted Yes must be allowed to become independent in their own right?

    Shockeroonie. Funny how "unionists" always turn out to be the real partionists.

    ReplyDelete
  36. Just leave it out or do separate companies. You can't replace the existing poll for the reason you've given

    ReplyDelete
  37. It has to be treated separately to the online polls. Regardless of the weightings later applied the raw data is coming from a quite different source.

    ReplyDelete
  38. James,

    Yes it was land line. They introduced themselves and asked if there was anyone in the house in the age 25-35. I said yes. They asked if they could speak to them, I said no, she's in Ibiza. He then asked my age and asked if I would take the survey.

    ReplyDelete