A little flurry of excitement tonight, because YouGov have released a table showing that a poll conducted between the 21st and 27th of January saw the anti-independence campaign's lead slump by 5% since the previous poll conducted between the 27th of November and the 2nd of December.
Yes 33% (+2)
No 52% (-3)
Just one snag, though - we previously knew nothing at all about any poll conducted between the 27th of November and the 2nd of December, nor any poll that contained the findings it supposedly showed. We did, however, know about a YouGov poll that was conducted more recently than that (between the 6th and 9th of December) with much better results for Yes, but that one is missing from the table.
So what the hell is going on? A possible clue is that the chart quotes the precise question posed to respondents, and to the unalloyed horror of anyone who cares about polling accuracy it contains the notoriously biased preamble designed (presumably by Peter Kellner) to coax people into reporting an anti-independence opinion. Unless YouGov have been lying to us, that preamble was finally dispensed with for their last two published polls of 2013, predictably producing a dramatic improvement in the Yes position. And yet we had plenty of anecdotal evidence that members of the YouGov panel were still being asked the referendum question with the preamble intact, which suggested that Kellner and co were continuing to use it for the No campaign's internal polls, or for unpublished 'testing' polls. My best guess is that tonight's news indicates that the latter was the case, and that the newly-released table is supposed to include only those polls that used the biased preamble. If that's true, the substantial swing to Yes suggested by the numbers above can be regarded as real, because any poll that used the biased preamble cannot be meaningfully compared to one that used a neutral question, and must instead only be compared to previous polls with an identical wording. It would therefore also imply that the No lead is lower than at any time since the preamble first reared its ugly head in early 2012.
All the same, it's (to put it mildly) absolutely bloody outrageous that Kellner is still seeking to distort the media coverage of this campaign with quasi-push-polling, and indeed with poll numbers that are casually published without any apparent regard for the standard BPC rules on transparency and disclosure. Unless we're allowed to see the datasets for this poll, I'd be inclined to say that academics and the media would be utterly wrong to regard it as being part of the 'true canon' of polling in this campaign. The only reliable aspect is the very favourable trend for Yes, but the headline figures themselves are hopelessly tainted.
The other news from this poll is that the SNP have retaken the lead on the Holyrood regional list vote, having apparently slipped six points behind in the unpublished 'phantom' poll conducted in late November/early December.
Maybe YouGov have made a mistake and it's this December poll they are referring to as the January one?
ReplyDeletehttp://d25d2506sfb94s.cloudfront.net/cumulus_uploads/document/2r7gjlmi92/YG-Archive-131209-Scotland.pdf
I don't think it can be, because the Holyrood figures don't match up. The only thing that makes any sense to me is that it's a chart including only those polls that used the biased preamble - although the one flaw in that theory is that it includes the September poll that was supposed to have been conducted without the preamble (unless there was a parallel poll using the preamble conducted on exactly the same dates and with exactly the same results, which is possible but unlikely).
ReplyDeleteIt can't be only the surveys that used that biased preamble, I'm looking at the chart and there are polls from all the way back from 2008 (although the preamble they used back then was biased too but it was phrased completely differently).
ReplyDeleteI thought maybe it was all polls they have ever carried out on independence for clients, but then why on earth would they display the biased preamble at the top of the chart and exclude the Sunday Times one from December? Maybe they just forgot?
It wouldn't surprise me if YouGov themselves don't even know.
Calum, the significance is that there is not one but two unpublished polls on that list, while one published poll conducted in between those two is excluded. That strongly implies to me that it's intended as a list of polls using a different question (ie. the biased preamble) than that used in the recent published polls. Yes, there are polls going back to pre-Dodgy Preamble days, but the difference is that they specifically acknowledge in the table (using an asterisk) that the wording in those polls was different. It's strongly suggested that every poll listed from 2012 onwards used identical wording (in the literal sense that can't possibly be true, of course, because the SNP government made a slight alteration to the wording of the referendum question last year).
ReplyDeleteOf course it's classic Kellner to practice to pretend that independence polling history didn't start until 2008 - for some reason he doesn't seem to like to acknowledge that YouGov was showing a clear lead for independence a couple of years before that!
Just thought you'd like to know James. Your article was as thorough and illuminating as ever, and yet for SOME reason (despite the poll being the main subject of the Smithson thread) trying to link your article from PB brought down the wrath of Smithson with the post immediately deleted accompanied by blustering nonsense trying (and failing) to justify it.
ReplyDeleteEmbargoes, polling and betting James. Eh? ;-) PB better hope nobody decides to take a closer look at that anytime soon.
I've just found the tables for this new poll, and it says the question was just "Should Scotland be an independent country?". That's confusing.
ReplyDeleteThanks, Calum - I've written a new post. The information is clearly incomplete and I'm still sure that a preamble was used (although I'm less sure of what that preamble was).
ReplyDelete