Monday, April 12, 2010

Polling sleights of hand

I don't really want to get into paranoid nit-picking about the Scottish press' treatment of the SNP in this campaign (although of course in the case of the Record there'll be no need for paranoia - they really are out to get us) but someone urgently needs to call the Scotsman and Scotland on Sunday on their recent blatantly selective use of opinion poll figures. The principle objectives seem to be to underplay the resilience of the SNP vote, and to convey the impression that the SNP and Tories are now "roughly level-pegging" in Scottish voting intention for Westminster. Evidence of the first objective came when they tried to have their cake and eat it after YouGov's recent revision of its Scottish polls, using new weightings that are more favourable for the SNP. In their reporting of the first poll conducted under the new system, Scotland on Sunday used the modified figures from the previous poll in calculating the percentage change in each party's support, thus showing no progress for the SNP. Which would have been perfectly fair enough - if they had explained that the previously published figures from the last poll had been erroneous, thus clarifying that the somewhat hysterical reporting of that poll in terms of the implications for the Nationalists had been grossly misleading. They conspicuously failed to do so.

Evidence of the second objective came a few weeks ago (before YouGov's methodological shift) in a report that explained that "a recent YouGov poll" had shown the SNP just one point ahead of the Tories. Quite true...except somewhat misleading, given that it wasn't the most recent YouGov poll, which just happened to show the SNP in a clear lead over the Tories. Just a sloppy error? Well, the conclusion of this article in today's paper does make me wonder. Once again, the most recent YouGov Scottish poll showing a solid second place for the SNP in Westminster voting intention doesn't seem to be good enough, so the paper casts around for convenient alternative figures - this time resorting to "an analysis of the Scottish responses to the last five UK wide YouGov polls". In other words, a back-of-the-envelope subsample aggregate. Now, I'm happy to stand by my often-expressed view that subsamples are not totally meaningless - looking at the pattern over time can give you a rough sense of what's going on. But even a long-term average of subsamples can never be as reliable as a properly weighted, full-scale Scottish poll - and since we have figures from a relatively recent poll of that kind available, what on earth are the Scotsman playing at?

No comments:

Post a Comment