A stupid (and possibly malicious) mythology about the referendum polls which must not be allowed to take root

I wasn't intending to post again until after Christmas, but there's an article on Political Betting this morning which I can't allow to pass without comment. It's written by ICM's former head of polling Nick Sparrow, and seeks to cast doubt on the reliability of polls that are conducted using volunteer online panels. That's fine in principle, because we've always known that the most which can be claimed about online polls is that they work reasonably well in practice, although they definitely shouldn't work in theory. But Sparrow's use of recent Scottish examples to illustrate his reasons for thinking that online polling may not even work in practice is, I'm afraid, utterly misconceived, and possibly even malicious in intent.

"What if that poll, you know the one that said “Yes” would win in Scotland, the one that panicked the whole political establishment into making wild promises for constitutional reform for us all was – how can I put this – wrong; the product of views expressed by people with stronger views and a more optimistic outlook than others? People who might be considered to be more likely to embrace a new vision of an independent future for Scotland, less concerned than others, for example that Scotland may not have a currency, or place inside the EU."

Yeah, and what if that poll was in fact bang on the money, at least to within the standard 3% margin of error? What if we had lots of supporting evidence for that from other polls, including from PHONE POLLS? You know, like the ICM and Ipsos-Mori phone polls that put the Yes vote on 49%, well within the margin of error of YouGov's 51%? Not to mention the face-to-face poll from TNS-BMRB that had Yes on 50%?

What if there are an awful lot of people out there who simply can't understand that polls are a snapshot of opinion at a particular moment in time, and not a "prediction" of how people will vote ten days later? What if, because of that elementary misunderstanding, a politically convenient mythology is doing the rounds in some quarters that the referendum was never in fact too close to call ("the Jocks were always too sensible for that self-government nonsense"), and that the perception that it was had been caused by a single "wrong" poll? And what if someone from the polling industry, who must know perfectly well that none of this makes any logical sense whatever, buys into that mythology (albeit in a deniable way, using weasel words) in order to pursue an agenda of his own? Wouldn't that be rather cynical and irresponsible?

The reality is that polls using all data collection methods (phone, face-to-face and online) were in almost total agreement by the close of the referendum campaign. They all overestimated Yes, but not by much. YouGov's own on-the-day poll overestimated Yes by just 1% - and that was a 2% drop from the previous day's poll. That's fully consistent with Yes support having reached somewhere in the region of the high 40s by the penultimate weekend (reported as 51% in one particular poll due to the margin of error), before slipping back - which ironically is likely to have happened mostly because of the "shock and awe" campaign of terror that was triggered by the London establishment's reaction to the very poll Sparrow is moaning about.

"What if online polls, comprising panellists with stronger opinions than others, being more optimistic and more volatile suggest in the run up to the next general election that the LibDems will be annihilated, UKIP and the SNP in Scotland are surging upwards, Farage and Salmond will be the new kingmakers and mould breakers? In the end the “Yes” campaign in Scotland did not do as well as predicted, but did it do better than it would have done if the polls had suggested the “No” campaign were always going to win comfortably? What if online polls over the next few months inflate UKIP and the SNP, thereby encouraging more voters to switch to them? In the end they may not do as well as predicted by some polls, but they may do better than they would have done had earlier polls not suggested they were on the march.

This means pollsters are not innocent observers of public opinion, but active participants in the political process; not only reporting public opinion but helping to shape it."

As already noted, the Yes campaign could well have done without the poll that put them ahead. That poll did indeed "help" to shape opinion, but in a way that was - sadly - contrary to what Sparrow is slyly suggesting. But his point about the general election polls is, if anything, even more absurd than his witterings about the referendum. Tell me, Nick - which poll was it that first suggested that the SNP were heading for a landslide next May? That's right - it was an Ipsos-Mori PHONE POLL. As it happens, that poll gave the SNP an enormous 29-point lead, which is considerably larger than the lead that any online poll has reported. YouGov themselves have been positively conservative in comparison to the phone poll - they've shown SNP leads of "only" 16 and 20 points respectively.

Has this guy even been paying attention?