Yes campaign draw level in electrifying poll from traditionally No-friendly firm TNS-BMRB

Hmmm.  I think at some point over the last few weeks we must have had that "game-changer" that the London media kept telling us we needed.  Did you get the impression they weren't really expecting it to happen, though?  Here is the extraordinary latest referendum poll from TNS-BMRB...

Should Scotland be an independent country?

Yes 50% (+8)
No 50% (-8)

As you'll have seen from the previous post, I wasn't expecting a swing on anything like that scale.  However, there's a very straightforward explanation for this surprise, which is that TNS-BMRB's fieldwork is nowhere near as far out of date as it usually is.  It started less than two weeks ago on the 27th of August, meaning that it took place entirely after the second leaders' debate on the 25th.  The other crucial significance of the 27th is, of course, that it was the first date on which people can realistically be expected to have started voting by post.  So we now have quite a significant body of evidence that, at the very least, No were not that far ahead when the referendum got underway.  And quite possibly, the contest was more or less even-stevens.

There's almost a sense of TNS-BMRB "coming home" with this poll, because although they've become known during the referendum campaign as one of the three No-friendly pollsters (alongside YouGov and Ipsos-Mori), it wasn't all that long ago that they were by far the favourite firm of independence supporters everywhere.  As recently as the late summer of 2011 they showed a small lead for Yes, which was hardly untypical for them - they had shown the odd plurality for independence before that, and even when they had the anti-independence vote ahead it usually wasn't by much. Something very peculiar happened in 2012, though, and TNS moved suddenly and decisively towards the No-friendly end of the spectrum - exactly why that occurred is one of the many, many baffling mysteries about polling during this campaign.   And yet here they are, back again as (at least temporarily) a more Yes-friendly pollster than even Panelbase.

This poll is obviously the first strong piece of evidence that the massive-scale swing to Yes is not something that is going to be confined solely to YouGov.  However, given the inexplicable extent of the disagreement between YouGov and Panelbase about the recent trend, the TNS figures still can't be taken as proof that we are going to see this hugely encouraging pattern across the board.  It seems to me there are now two plausible possibilities - either a) there is going to be a straight split down the middle among the polling firms, with ICM, Survation and Panelbase (all of whom have relatively similar methodologies) not showing as big a movement towards Yes as the others, or b) Panelbase will prove to be the outright odd-one-out, with all the others showing a TNS/YouGov-type swing.  The latter possibility would constitute a strange turn of events, but far from an unprecedented one - during the winter Panelbase bamboozled us all by producing a swing to No (albeit within the margin of error) at a time when there was otherwise a broad consensus that Yes were making significant progress.  It could be that there's something a bit unusual about Panelbase's sample - maybe they've been overpolled, or maybe they were just especially politically committed from the outset, and are thus less likely to be open to persuasion than voters out there in the real world.  It's certainly striking how stable Panelbase's numbers have been throughout this campaign - yes, they've shown a modest drift towards Yes this year, but nothing on the scale that other firms have reported.  (And that was the case even before the drama of recent events.)

Let's stick with the point about 'the real world', because that's what makes both TNS-BMRB and Ipsos-Mori special in this campaign - in their own very different ways, they actively seek out a completely fresh sample for every poll.  If it wasn't for them, we'd be completely reliant on pollsters who use volunteer online panels, with all the danger that entails that the people being interviewed are not truly representative of the electorate at large.  Until now, it's been a matter of considerable concern that the only two non-online companies were so firmly on the No-friendly end of the spectrum, so a telling breakthrough for Yes in a TNS poll effectively counts double.  Indeed, because of what Panelbase are showing, we can't completely exclude the possibility that we've now swung to the other extreme, and that in the closing days of the campaign 'real world' polls (assuming there are more to come) will show a slightly more favourable position for Yes than the volunteer online panels do.  Admittedly I don't think that's at all likely, but it's no longer inconceivable.

Of course there's still room for doubt over whether a real world pollster will always produce representative results - for example, Ipsos-Mori might be missing crucial sections of the population due to their apparent practice of only phoning landlines. That isn't a concern for TNS who go out and knock on people's doors, but there's still the issue of how they weight their raw data.  It's striking that there's never a big enough gap in their raw sample between people who recall voting SNP in 2011 and people who recall voting Labour.  Although there's evidence from other pollsters that people are much better at accurately recalling how they voted in 2011 than how they voted in 2010, it's hard to escape the worry that there might be at least a modest degree of false recall at play among the TNS respondents, in which case the adjustment made to bring the sample into line with the actual 2011 result may be artificially flattering the Yes vote.  Fortunately, however, the adjustment in tonight's poll isn't as big as in one or two previous TNS polls, so even if Yes are being overstated due to this issue, it won't be by much.

And we also have to bear in mind the crucial weighting that TNS are failing to apply - namely country of birth.  It's mainly among online pollsters that we have evidence that there is almost always too great a number of English-born respondents in the raw data, but if by any chance this general rule also applies to non-online firms, then a weighting to correct the error would probably be sufficient to push Yes into a clear lead in tonight's poll.

Although Panelbase may be in disagreement with TNS and YouGov about the overall trend, the one thing all three firms agree on is that the gender gap has narrowed of late.  Even a consensus of three polls doesn't represent proof that it has actually happened, but it's certainly beginning to look very much like a real and potentially highly significant pattern.

The No camp still have the tiniest of leads in this poll on the unrounded figures, and indeed they have a lead even on the rounded figures when Don't Knows are taken into account (39% to 38%).  But as always with TNS, it's possible to apply various different turnout filters to the voting intention numbers, and there's one filter which actually puts Yes into the lead...

Whole sample (equivalent to 100% turnout) :

Yes 49.6% (+8.0)
No 50.4% (-8.0)

Whole sample excluding only definite non-voters (equivalent to 97% turnout) :

Yes 49.7% (+6.8)
No 50.3% (-6.8)

Respondents who say they are certain or very likely to vote (equivalent to 91% turnout) :

Yes 50.2% (+5.6)
No 49.8% (-5.6)

Respondents who say they are certain to vote (equivalent to 84% turnout) :

Yes 49.9% (+4.7)
No 50.1% (-4.7)

For the first time with TNS, it's the figures for definite voters that look most likely to correlate with the actual turnout.  But given the huge jump in people's reported likelihood to vote in this poll, you do begin to wonder if a 90%+ figure is just about possible.  After all, it happened in Quebec...

* * *

REQUIRED SWINGS

Swing required for 1 out of 6 pollsters to show Yes ahead or level : 0.0%

Swing required for 2 out of 6 pollsters to show Yes ahead or level : 0.5%

Swing required for 3 out of 6 pollsters to show Yes ahead or level : 2.0%

Swing required for 4 out of 6 pollsters to show Yes ahead or level : 3.0%

Swing required for 5 out of 6 pollsters to show Yes ahead or level : 4.5%

Swing required for 6 out of 6 pollsters to show Yes ahead or level : 7.0%

* * *

SCOT GOES POP POLL OF POLLS

It's getting to the stage where I have to put a significant health warning on the Poll of Polls.  As it now seems likely that there has been a substantial swing since the second leaders' debate, the fact that a full third of the Poll of Polls sample is made up of polls conducted before that debate (Ipsos-Mori and ICM) means that the numbers listed below are likely to be underestimating the true strength of the Yes vote.

MEAN AVERAGE (excluding Don't Knows) :

Yes 47.0% (+1.1)
No 53.0% (-1.1)

MEAN AVERAGE (not excluding Don't Knows) :

Yes 41.5% (+1.0)
No 46.8% (-1.0)

MEDIAN AVERAGE (excluding Don't Knows) :

Yes 47.2% (+1.5)
No 52.8% (-1.5)

(The Poll of Polls is based on a rolling average of the most recent poll from each of the pollsters that have been active in the referendum campaign since September 2013, and that adhere to British Polling Council rules. At present, there are six - YouGov, TNS-BMRB, Survation, Panelbase, Ipsos-Mori and ICM. Whenever a new poll is published, it replaces the last poll from the same company in the sample. Changes in the Poll of Polls are generally glacial in nature due to the fact that only a small portion of the sample is updated each time.)