Are Peter Kellner and YouGov Yes Scotland’s hidden asset?

A guest post by Alasdair Stirling

YouGov's Peter Kellner has written an extraordinary blog article entitled 'Why Do the Polls in Scotland Vary So Much?' in which he puts forward what Sir Humphrey might call a 'brave' proposition. The gist of Kellner’s view is that the Scottish independence referendum opinion polls (other than YouGov's of course) fail to capture the true character of the constituency of Scottish voters that supported the SNP in 2011. He contends that, absent a corrective methodology such as YouGov applies, polling samples will contain too many of what he terms 'Passionate Nats' and too few – again his term - 'Passing Nats' and that that the resulting imbalance skews the forecast Yes/No percentages in favour of Yes to independence.

Now the SNP is not unique in finding its support from both committed loyalists and floating voters; all political parties experience this phenomenon so it might come as a surprise that Kellner does not advocate any similar corrections for poll respondents that supported the other Scottish political parties in 2011. It is not hard to see a rationale for this approach as it may be arguable that changes in voter allegiance amongst the Unionist parties will have little effect on the the binary Yes/No independence voting intention.

However, Kellner is in some difficulty with his policy of correcting only the poll respondents supporting the SNP in 2011 when it comes to forecasts of party support vis-a-vis the 2016 Scottish general election. Clearly if all of the Scottish parties are susceptible to this problem but the correction is applied to only the SNP supporting respondents, then it is likely that the forecast levels of party support for the 2016 election will be wildly inaccurate. Accordingly, by deduction we have to conclude that Kellner believes either: that the problem applies only to the respondents that supported the SNP in 2011; or that in so far as it applies to all parties its effect is only statistically significant as regards respondents that supported the SNP.

Kellner uncovers this phenomenon – lets us call it the 'Kellner Effect' – by investigating the voting record of 2011 SNP supporting respondents and cross-referencing the same respondents' 2010 Westminster election voting record (YouGov of course rely more on their panel data than voter recall). Kellner advocates that SNP supporting poll respondents be disaggregated into two constituent groups (loyalists and floaters or in his terms 'Passionate' and 'Passing') sampled according to quotas established by reference to the 2010 election and then re-aggregated by means of (if necessary, aggressive) political weighting. Kellner, without explanation, limits the application of this methodology – lets call it the 'Kellner Correction' – to SNP loyalists and Labour floating/tactical voters and concludes his blog article with the assertion that any poll that does not apply the Kellner Correction is wrong (and we must suppose just so much toilet paper).

In his UK Polling Report blog Anthony Wells gives a fine explanation of the sampling and weighting methodologies used by the polling industry. In regard to sampling, he points out why a truly random sample is impractical and cogently explains the Quasi-random and Quota sampling techniques that the polling industry uses to achieve viable polling samples before going on to explain the use of weighting to correct discrepancies and bring a viable polling sample more completely into line with the nature and character of the broader community being sampled.

Quasi-random and Quota sampling techniques are therefore the very foundation stones of the modern polling industry and the Kellner Correction can only be necessary if normal Quasi-random and Quota sampling techniques are unable to capture the true character of the community of SNP voters in Scotland. Absent some malign manipulation of the polling process, which Kellner does not suggest or imply, we must take his blog article as challenging the foundations of political polling. For if the Kellner Effect can apply to SNP respondents in Scotland, why not Conservative/Labour/Liberal/UKIP respondents throughout the UK; and if so, where when and why? Can we reasonably consider the merits of any poll (certainly any Scottish poll) that does not rigorously apply the Kellner Correction?

However these considerations are just the start. The broader implications of Kellner’s Effect and Correction are nothing short of staggering! If Kellner is right, then any polling company not applying the Kellner Correction can effectively shut up shop (at least in Scotland) and YouGov are positioned to establish a monopoly of political polling in a revalidated United Kingdom whilst the rest of the industry plays catch-up. On the other hand, if he is wrong it is hard to see that Kellner himself (having allowed his company to depart so fundamentally from the industry’s tried and tested methods) can survive an independence referendum result that falls into line with the forecasts of the polling companies not employing the Kellner Correction. But beyond potentially ruining his own career, Kellner’s is also throwing the dice for YouGov itself as we can only contemplate the detrimental effect on its business if events prove to its customers that YouGov lacks a commitment to the industry’s tried and tested methods, impartial approach and rigorous standards.

Notwithstanding, perhaps the greater implications lie in the referendum debate itself. With other polling companies showing a narrowing of the forecast Yes/No result, the Better Together campaign take much succour from YouGov’s consistently strong forecast of an emphatic No vote (and its effect on the many Poll of Polls type averages). Better Together’s Campaign Director Blair McDougall (not to mention Labour activist Ian Smart) have much tongue-in-cheek fun tweeting ‘no room for complacency’, but if Kellner is wrong then Better Together really do have no room for complacency, but will thanks to Kellner remain blind to the need to alter strategy. So in addition to imperilling his own career and business, Kellner may yet turn out to be massively assisting Yes Scotland in bringing about the end of the United Kingdom.