It was fairly obvious from the hints being dropped on Twitter last night that a new Panelbase poll on the independence referendum was on its way, and that it was far more favourable for Yes than yesterday's YouGov poll (which we now know was indeed tainted by using a leading question that painted independence in a pejorative light). However, I couldn't have imagined quite how favourable it would prove to be -
Should Scotland be an independent country?
Yes 44% (+7)
No 43% (-3)
This is the first time that the pro-independence camp have been in the lead with any pollster since a TNS-BMRB poll in the late summer of 2011 put Yes ahead by one point.
So what can possibly explain the extraordinary divergence between two polls published in successive days, one which puts Yes one point ahead, and another which puts No thirty points ahead? Methodology, obviously, but what in particular? It's easy to home in on the outrageously leading preamble that YouGov use when posing the referendum question, but as I've said on previous occasions that's unlikely to be anything more than a small part of the explanation - if I was going to hazard a guess, I'd say that it probably doesn't make more than 2% of a difference. Nevertheless, as a matter of principle it's extremely bad practice for YouGov to continue behaving in this way, and it's high time that greater pressure was brought to bear on them, both from their peers and from psephologists. In case you're wondering, the Devoplus campaign that commissioned the poll almost certainly aren't to blame in this particular regard - as far as I can see, the preamble is identical to the one YouGov have used on several previous occasions. Peter Kellner just seems to have a bee in his bonnet that he can 'improve' on the actual, Electoral Commission-approved referendum question.
If Professor John Curtice was prepared to go out of his way to criticise the recent WoS poll for using leading wording in one particular question, then he really has no excuse left for not issuing a warning to YouGov that their polls will have a question mark hanging over them for as long as they continue to steer their respondents in such a blatant way towards giving an anti-independence response on the headline question.
* * *
Undoubtedly the daftest comment about today's poll was made by former "senior Labour MSP" George Foulkes - but did he do it deliberately? He claimed that it was "astonishing" that the Courier had led on the results of a poll conducted by an organisation that pays its online panel for responses - apparently unaware that all online pollsters use a paid panel, including YouGov, who carried out the poll that George's campaign were quite content to see Scotland on Sunday lead with yesterday. When this rather obvious double-standard was pointed out to him, George mumbled something about YouGov being an "established" pollster. Oh-kaaay. So the difference in credibility is that YouGov merely pay their respondents in an "established" way? How does that work?
* * *
A quick appeal - does anyone reading this have an account at Political Betting? If so, I'd be grateful if you could write a quick comment there about the sensational news from the Panelbase poll - I can't see any reference to it so far, and I'm obviously not in a position to put that right. The reason it might be important to do so is that some London journalists still seem to take their cue from PB when summarising the lie of the land in polling terms. Inexplicable, but there it is.
Even if you don't have an account, it would only take a minute or two to set one up (although your first comment might have to go through pre-moderation).
* * *
UPDATE (6.30pm) : At last - John Curtice speaks out on YouGov's dodgy preamble...
"Conversely, at the other end of the spectrum YouGov’s poll for Devo Plus described the referendum as a ballot on ‘Scotland leaving the United Kingdom and being an independent country’ a description that might have been thought capable of discouraging some respondents from saying Yes."
i say great keep this up and we will be running cant wait i want to celebrate
ReplyDeleteOver on pb.com the panelbase poll has been rubbished on the basis that the voting intention question was not asked first, and so the earlier questions will have affected the result.
ReplyDeleteI can't find any source for the data tables for this poll - or any previous panelbase poll - but it would be an issue if it were the case.
I think we can safely say that any poll showing a Yes lead would have been rubbished on PB, but I'm glad to hear that it's at least been noticed now. I presume that none of the usual suspects think that YouGov's preamble is an issue at all?
ReplyDeleteTo answer your question, the datasets are out, and the main referendum question is listed third. I'm not sure whether that means all respondents were asked the question third, or whether the order was selected in a random order for each respondent (as was apparently the case for the recent Panelbase poll commissioned by Wings over Scotland). In any case, it's worth making the point that it's by no means standard to ask the referendum question first - typically both Westminster and Holyrood voting intentions will be asked for first, which theoretically could skew the outcome.
I agree that, since the referendum is now the most important upcoming vote, it would be best practice to always ask that question first. But this poll is scarcely unique (or even unusual) in failing to follow that best practice.
It's not a bias unique to pb.com to accept something because it confirms prevailing opinion, but to find a reason to rubbish evidence that questions the existing orthodoxy.
ReplyDeleteThe Yougov question, with its bizarre preamble, has made an appearance, but the usual suspects are more interested in the announcement by Malcolm Bruce that he will not contest the next GE, when it comes to matters Scottish.
There's a topic on it up on PB now James.
ReplyDeleteYou'll hardly be surprised that the most dotty and comically inept tory spinners are furiously trying to rubbish the Panelbase result. (despite them parroting the yougov as if it were the last word on the matter of course) I posted a link here with some of what you said as it covered the main bases.
TBH I haven't been on there in a couple of days as it's even more of a right wing zoo than usual, which is saying something.
There was the obvious hilarity of Cammie and Hague blundering about on Syria lately, but needless to say that was everyone's fault but theirs according to the PB tory spinners.
You covered it well in your article. The Blairites warmongers never seem to realise the damage they do to their cause as they cheer on the missile strikes.
UnionDivvie more than likely mentioned the Panelbase as well I would think.
Today as I went to meet other Pensioners on our trip out to Pitlochry. I noticed on the Courier news boards outside the newsagents 'New Poll has Yes Campaign in Front'. I nearly did cartwheels. Still I didn't buy the paper. 2 of my travelling companions in the car I was in mentioned the poll and why they have moved away from No to an almost Yes. I just listened to what they said and didn't think I needed to say anything more. The information will be passed on.
ReplyDeleteGreat stuff, Marcia. I've heard there may be more polls later this week, and it's probably too much to hope that they'll be quite as good as this one, but time will tell.
ReplyDeleteMick : Thanks very much. I did check back later in the afternoon and spotted that TheUnionDivvie had posted, and I may well have missed other comments earlier in the morning. I'm not too concerned about the predictable attempts to rubbish the poll - just so long as the London media establishment are fully aware of its existence.
I can honestly say that I haven't felt the slightest urge to visit PB over the last few weeks, let alone post there. But the first poll lead for Yes in two years is a bit of a special occasion!
It beggers belief that I agree with Prof. Curtice’s point about Panelbase using initial questions before asking the ‘main’ referendum question, and his belated recognition of the possible effect of the YouGov preamble. That said, it beggars belief that he could think that the effect of Panelbase asking the initial questions in the SNP’s most recent poll could bring about a 30 point shift in opinion. Surely, the real reason for the divergence of the polling results is the effect of the political identification and/or voting intention methodology on populating the polling sample?
ReplyDeleteAlasdair, in the PB piece that Mick mentioned above, they trotted out (for the millionth time) the famous Yes, Prime Minister clip as an illustration of how the Panelbase question order could supposedly have affected the results. But that was actually a terrible example, because as Scottish Skier pointed out in a Wings comment, the preliminary questions that Panelbase asked weren't actually leading. There was no equivalent of a "Are you concerned about the effect of arming teenage boys with guns?" question that can only have one possible answer.
ReplyDeleteJames,
ReplyDeleteI see that a few people are tweeting that there is another panenbase poll coming out tomorrow to shows the No side with only a 1% lead. That is good news as it within the margin or error and confirms the poll of yesterday. I'd rather have the No side slightly ahead at this moment in the long campaign.
I'm wondering if that might have just been a rumour, Marcia - all I can see at the moment is people talking about the publication of further questions from the same Panelbase survey. Oh, and there's our old friend Mr Smithson smugly announcing his foreknowledge of an "interesting" referendum poll tomorrow showing "markedly different" results from Panelbase. Well, it would have to be markedly different before he would take it remotely seriously, wouldn't it?
ReplyDeleteI wouldn't be too worried about a substantial No lead if it comes from Ipsos-Mori or TNS-BMRB, because that would be in line with their previous polls - what really matters is the trend.
Surely, the real reason for the divergence of the polling results is the effect of the political identification and/or voting intention methodology on populating the polling sample?
ReplyDeleteMore likely in my opinion is that a large number of people simply haven't made up their minds, and are possibly easily swayed one way or the other by whether it is raining or not.
That would leave it all wide open to be won or lost in the campaign itself.