Tuesday, March 21, 2017

CONFIRMED : Last night's ludicrous Sky "poll" was not weighted by EU referendum vote

As you may have seen, Sky News made an exasperating attempt last night to distort the referendum debate with a wildly unreliable online "poll", conducted by themselves and using their own paying customers as respondents, which purported to show that Theresa May is significantly more popular in Scotland than Nicola Sturgeon - a finding that is light-years out of line with recent properly-conducted polls from firms affiliated to the British Polling Council. 

The numbers were so patently absurd that I didn't think they even warranted the effort of a blogpost, although I did post a tweet pointing out that it was a junk poll and should be disregarded.  That prompted a reaction this morning from Harry Carr, head of "Sky Data", who insisted that it was a nationally representative poll.  He also produced datasets which showed that some weighting had been done - but there was no obvious sign of many of the political weightings that are standard in Scottish online polls, such as by recalled indyref vote, recalled EU ref vote, or recalled Holyrood vote.

As the poll bears all the hallmarks of having far too many Leave voters in the sample, I asked Harry a simple question - had he weighted by EU ref vote?  His answer : No.  He had weighted by recalled 2015 general election vote, but not by EU ref vote.  I strongly suspect that also means he didn't bother weighting by recalled Holyrood vote or by recalled indyref vote.

There's your explanation for the poll's nonsense results right there.  A telephone or face-to-face poll can possibly get away without doing proper political weighting, and just relying on demographic weighting.  But an online poll - no chance.  Online polls are different because you know in advance that you're drawing from an unrepresentative pool of potential respondents - Sky customers, for example, may well have political leanings that are different from non-Sky customers, and you have to carefully weight to correct for that.  I literally can't think of a single British Polling Council firm that would have attempted to conduct an online poll in the way that Harry has done.  It also looks like he didn't bother weighting by country of birth - a failing that he has in common with only BMG.

I stand by my original assessment - junk poll, ignore.

By the way - here's a challenge for Harry.  Conduct a Scottish voting intention poll, and publish it even if it puts the Tories in the lead.  Then try to keep a straight face when you defend your methodology.  Go on, I dare you.

16 comments:

  1. Thank you, James - you've come up trumps again. I've been waiting for your rebuttal of that ludicrous poll, and will make the most extensive and effective use of it that I can.

    ReplyDelete
  2. There was once a time when socialism basically meant trying to improve the wages,conditions,etc of the working man,all admirable and still required today.This however no longer exists all we have now is identity politics ie i am a nationalist,i am gay, iam black,i am disabled etc.Its all a distraction and causes divide and conquer,lets get back to unity for the working man.

    ReplyDelete
    Replies
    1. Unity for the working man under a hard-right Tory brexit britain whose closest ally is Donald Trump? Aye, what could go wrong?

      Delete
    2. The identity politics of the BritNatsMarch 21, 2017 at 6:04 PM

      "Can't we just jump to @RuthDavidsonMSP being First Minister?"

      - Ian Smart Scottish Labour Spokesperson, member, TV pundit and British Nationalist

      Delete
    3. Indeed the Scottish Nat sis who played no part in the class struggle have by default conned their way into power. They are by nature pariahs.

      Delete
    4. Seek professional help.

      Delete
  3. It will be interesting to see who May addresses on the visit to her Scottish region.
    If she is so popular,it should be a huge audience or will it be an Osborne quick press call and then back to the airport?

    ReplyDelete
  4. It's as reliable as an on-line poll of "Wings" subscribers.

    ReplyDelete
    Replies
    1. Clearly.

      Did anyone suggest an online poll of 'Wings subscribers' only would be more accurate?

      Straw man bollocks.

      Delete
  5. Thanks James. I was sent the poll and completed it. I wish I had screenshot the questions. One thing that does strike me is that 'approval' is being presented here and I was not asked if I approved of anyone but whether or not I thought certain people were doing a good job. No mention of a good job of what. I did give Ruth Davidson a higher than rubbish rating because I think she is quite good at being an opportunistic pathogen for example. But I was not asked if I approved of anyone!

    ReplyDelete
  6. Sly News. Junk poll from a junk broadcaster.

    ReplyDelete
  7. I've seen this tweet (my capture as at 21Mar, 10:46pm) but didn't manage to trace from this to find out how the 185 and 2200 figures were obtained.

    Down In The Abyss‏ @Thedee1893
    So this @SkyNews poll about May/Davidson doing brilliant in Scotland interviews 2200 people of which 185 were Scottish 😂😂😂😂😂
    RETWEETS 28 LIKES 17

    ReplyDelete
  8. Yeh, yeh, yeh all polls with anti-independence findings are unreliable dog poo, and all that show pro-independence support are reliable six-sigma statistically-valid guarantees

    That's after all why independence is now at 273% in the polls after increasing it's lead in all 376 reliable polls since 2008 eh james?

    ReplyDelete
    Replies
    1. We Said Remain to Leave Men, and Remain Means RemainMarch 22, 2017 at 7:41 PM

      This poll doesn't have an anti-independence finding - it didn't ask about independence, but about people. The reason the poll has no credibility is because of its methodological flaws, which I identified in the blogpost. Do try reading the thing before shooting your mouth off, there's a good chap.

      Delete
  9. Thing is James inspite of all the polls you Nat sis are not well in the lead. And this farce over brexit is just a last gasp attempt to hold a referendum. You nat sis have no respect for the majority. Declare UDI and we can fight. At least the would be honerable.

    ReplyDelete