Friday, May 24, 2013

On grandiose claims about self-selecting polls

I've got nothing against self-selecting internet polls - I've run a few myself over the years, and they can be quite entertaining if a lot of people take part. But at the end of the day they are unweighted, unscientific voodoo polls, and even if several thousand people took part they wouldn't tell you a thing about the true balance of public opinion. They are, in the immortal words of Peter Snow, "just a bit of fun".

Last September, Subrosa ran a self-selecting poll on her blog about whether an independent Scotland should be an EU member or not. The results were as follows -

Yes - 110 votes
No - 176 votes
Stay in the UK but leave the EU - 72 votes
Unsure - 29 votes

(The other obvious option of staying in both the UK and EU does not seem to have been provided.)


Both at the time and then again yesterday, Subrosa made quite extraordinary claims about the significance of those results. I challenged her on both occasions, pointing out that bloggers tend to attract readers who agree with them, and she was kind enough to respond. I must say, though, that I found her responses somewhat baffling. Here was the first one in September -

"Yes James, I am anti-EU and that will continue. Many of my readers disagree with me about lots of issues and I respect their opinions.

If you're suggesting that this poll is invalid because it's turned out to be 50/50 then I think you're wrong.

In no way am I stating the poll is an accurate reflection of the whole of Scotland but at least it proves that 176 people don't want an independent Scotland in the EU - if nothing else. :)"

The first couple of sentences are irrelevant because I hadn't suggested there was anything wrong with an individual continuing to be anti-EU, nor had I disputed that some (ie. "many") of her readers disagree with her.

The third sentence is peculiar, because self-evidently the poll hadn't turned out to be 50/50, nor had I suggested that was the reason for it being invalid.

The fourth sentence is technically accurate, but so what? The fact that 176 people hold a particular viewpoint is spectacularly unimportant in the overall scheme of things. Even if we assume that every single one of them was a Scottish resident (almost certainly not the case), that would constitute a mere 0.00003% of the population.

And here was her response to me yesterday -

"A great majority of the readers of this blog support independence James and have done for many, many years. It's quite strange that you're implying the majority are against the EU because I would disagree.

What was interesting about my tiny poll was the number of people who were undecided about the EU yet continually we're told that most of us support it.

Believe me not all my readers agree with me; in fact many take the trouble to tell me just how strongly they disagree and I respect their views."

First of all, I didn't "imply" that the majority of Subrosa's blog readers are anti-EU - I was simply looking at the poll results which clearly suggested that was the case. And was the number of undecideds (a mere 29 people) really the most interesting thing about the poll? How exactly does that contradict the idea that "most of us" support membership of the EU?

And once again, I hadn't disputed that some of her readers disagree with her - I simply pointed out that it was unsurprising that the majority do not.

What am I missing here?


  1. Just to say, as a country, as both you and Sub Rosa seem to agree we need to be, early doors, we cannot reach our own decision on the EU until after independence.

    I don't see the point in even thinking about it until after we vote 'Yes'.

    What would be the point?

    Wouldn't you say Sub Rosa's 176, or whatever it was, will be completely subsumed by whatever the English majority decide?

    I agree completely with Sub Rosa on independence. I think she and I will be on opposite sides of the debate after that. But that is good.

  2. Subrosa is a UKIPpper in every sense except that she wants Scotland out ot the UK. All the rest of the evil fascism is still there. Same goes for the mentalcase comments.

  3. Anonymous

    "Subrosa is a UKIPpper in every sense except that she wants Scotland out ot the UK. All the rest of the evil fascism is still there. Same goes for the mentalcase comments."

    Especially the anonymous ones, eh?

  4. For the avoidance of doubt, I don't agree that Subrosa is an "evil fascist". She's always been extremely friendly and helpful towards me. I don't even have a problem with her views on Europe, even though I profoundly disagree with them. I was simply making a very narrow point about her misuse of poll figures that have no validity whatsoever.