Sunday, October 28, 2012

Why are YouGov so convinced they can "improve" on the referendum question?

I overlooked this YouGov poll on independence the other day, which essentially shows an identical position to the previous poll by the same company a week earlier -

Yes 29% (-1)
No 55% (-1)

(Note : In my post about the earlier poll, I incorrectly gave the figure for Yes as 31% rather than 30%, because the question had been asked two different ways with slightly different results.)

A supplementary question asked whether Yes Scotland (the Yes campaign) or Better Together (the No campaign) had produced better evidence to support its case. The results were strikingly different -

Yes Scotland 33%
Better Together 32%

One point stressed in the reporting of this poll is that the main question asked by YouGov was the actual referendum question proposed by the Scottish Government. This would indeed constitute progress for a polling company that has for years much preferred questions like "Do you REALLY think that Scotland should be WHOLLY separate from its dear brothers and sisters in the rest of the United Kingdom, cast adrift in the North Atlantic without food, shelter or warmth?". But, alas, the story isn't quite so simple. YouGov apparently still believe their panel are far too thick to understand the meaning of the words "Do you agree that Scotland should be an independent country?", and felt obliged to offer an 'explanatory' preamble to the question, pointing out (you've guessed it) that what the referendum is really about is Scotland leaving the United Kingdom.

There's no great mystery as to what's going on here. Peter Kellner, the head of YouGov, recently penned an article in which he once again lambasted the SNP for proposing a 'loaded' question. Well, he's perfectly entitled to that faintly ludicrous opinion, but what he isn't entitled to do is allow his personal beliefs about what would constitute a "better" question to continue to skew his polls, however subtly. It should surely be self-evident that the most robust, credible and reliable polls will be those that ask the actual referendum question in a straightforward, unadorned manner.

Incidentally, Kellner's article also contained the jaw-droppingly hubristic claim that he knows - literally knows for a fact two years in advance - that there will be a No vote in the referendum, a conviction for which he offered a string of spurious justifications. Now, it would be extremely easy to dismiss this as a partisan stunt by a known Labour sympathiser (and the husband of Baroness Cathy Ashton), but in truth I think that Kellner is a serious analyst who just has some pretty major blind spots. Anyone who watched the replay of the BBC's 1992 election coverage earlier this year will have seen him make a series of utterly bizarre predictions - one was that Brussels would soon somehow 'force' the UK to use proportional representation for its national elections. Twenty years on, it's sad to say that his bolder predictions are no more soundly-based.


  1. Quite. Didn't their polls and their competitors predict that Iain Gray is be our FM by any chance?

  2. You'd have thought that by now there would be no explanation needed as to what the question referred to, and therefore no need to, while using the question preferred by the government, adding a few emotive words like "separation".

    But then, that would get the answer one was looking for, would it.

    I didn't know he was married to Ms Ashton. I must look closely at the next questions about how useful the EU foreign secretariat is...

  3. I doubt if the preamble made that much difference to the poll findings, Tris - maybe a point or two at most. But it's still bad practice, and it's a particularly sneaky way of doing it, because it allows them to innocently claim that they're now asking the real question.

  4. If the preamble made no difference then they wouldn't do it.

    Would anybody admit to voting labour if the preamble was about iraqi corpses, 90 days detention, massive debts etc? And would labour stooge kelner even allow harm to his paymasters?

  5. I think YouGov would say it was done for "clarity"!

    But it's not the business of a pollster to shape the trajectory of an electoral race, or indeed to shape the terms of a debate, which seems to be Kellner's mission at the moment. Their business should be to provide the most accurate possible snapshot of what would happen if the referendum was held now - and that means using the real referendum question without providing helpful unionist 'guidance' to respondents.

  6. The tentacles of 'The Establishment' are well seated and will come from all directions especially those in unexpected circles.