Recalculating the Poll of Polls and long-term trend figures with Angus Reid excluded

First things first - Survation have released the datasets for yesterday's referendum poll, and they show that on the unrounded figures (or rather the figures rounded to one decimal place), the No lead has indeed fallen to the lowest level that the firm has reported so far -

In the referendum, voters will be asked, "Should Scotland be an independent country?" If this referendum were held today, do you think you would vote Yes or No?

Yes 38.4% (+1.4)
No 46.1% (-1.1)

With Don't Knows excluded, the position is -

Yes 45.4% (+1.5)
No 54.6% (-1.5)

The headline No lead now stands at 7.7%, compared to 8.9% in February, 8.3% in March, and 10.2% earlier this month. So the idea that is being put around that Survation have failed to show any tightening of the lead in recent weeks is a touch misleading. It's true that the changes since February could very conceivably be normal margin of error variation (although that is emphatically not the case in respect of the changes since the Survation poll conducted in January), but in my view it's always significant when a poll produces figures that are outside the range we've seen before, as this new poll has done. It could indicate a tightening of the lead, but even if it doesn't, it would instead indicate that the 'static' position is a touch better for Yes than we previously realised. When taken in combination with the sensational ICM poll that was conducted at the same time, I'd suggest a genuine tightening of the lead is the more probable scenario in this case.

After a recent poll from another firm, Calum Findlay pointed out the proportion of Yes voters who had voted SNP in 2011, which I think is quite an interesting way of looking at things. In this poll, only 231 of the 374 respondents who said they were voting Yes were SNP voters in 2011. That's significantly fewer than two-thirds. It's sometimes said that "if Yes are to have ANY chance of winning, they'll have to extend their coalition of support EVEN BEYOND the HUGE numbers who voted SNP in 2011", as if that's the hardest thing in the world to do, but we can see from these figures that Yes have already done just that. If they had also claimed the allegiance of everyone who actually did vote SNP three years ago, they'd effectively already have the referendum sewn up - but, alas, it doesn't work like that. We're talking about two very different coalitions of support.

A few other interesting nuggets -

* The No lead is a touch on the high side among 16-24 year old voters, who as usual have been massively upweighted (from 62 real people to 121 'virtual' respondents). So if Yes are being underestimated among the young due to normal sampling variation, that could be slightly distorting the overall headline figures in a No-friendly direction.

* A healthy 26.2% of people who voted Labour in 2011, and 18.6% of people who voted Liberal Democrat, are planning to vote in favour of independence. Those figures rise to 31.3% and 19.3% respectively when Don't Knows are excluded.

* Of the very small number of people in Scotland who are planning to vote UKIP at the next Westminster general election, more than a third are also planning to vote Yes to independence in September. Does Nigel Farage really want this kind of riff-raff inside his tent?

* * *

SCOT GOES POP POLL OF POLLS

When I started the Poll of Polls a few months ago, I always knew that the list of pollsters represented in the sample would not remain static. I expected that new firms would enter the fray (as has happened with Survation), but what I didn't really anticipate was that any firms would effectively drop out. That appears to be what has happened with the Canadian firm Angus Reid, who haven't conducted a poll since August. Until now, I've left their most recent poll in the sample in the hope that they might pop up again and make things simpler for me, but it's getting to the point where this is having a significantly distorting effect on the median average. So the time has come to remove Angus Reid (although you can guarantee that now I've done this they'll be bound to produce a new poll next week!).

MEAN AVERAGE (not excluding Don't Knows) :

Yes 35.8%
No 47.2%

MEAN AVERAGE (excluding Don't Knows) :

Yes 43.1%
No 56.9%

MEDIAN AVERAGE (excluding Don't Knows) :

Yes 43.4%
No 56.6%

(The Poll of Polls is now based on a rolling average of the most recent poll from each of the pollsters that have been active in the referendum campaign since September 2013, and that adhere to British Polling Council rules. At present, there are six - YouGov, TNS-BMRB, Survation, Panelbase, Ipsos-Mori and ICM. Whenever a new poll is published, it replaces the last poll from the same company in the sample. Changes in the Poll of Polls are generally glacial in nature due to the fact that only a small portion of the sample is updated each time.)

I'm hugely indebted to Ivan McKee of Business for Scotland who retrospectively recalculated the Poll of Polls without Angus Reid all the way back to September, and sent me his table. So it will still be possible to chart the long-term trend, which now looks even more dramatic.

The No campaign's lead in the Poll of Polls headline figures (past updates recalculated to exclude Angus Reid) :

Sep 2013 - 21.6%
Sep 2013 - 21.4%
Sep 2013 - 19.4%
Oct 2013 - 18.8%
Oct 2013 - 18.4%
Oct 2013 - 18.2%
Nov 2013 - 18.4%
Nov 2013 - 18.0%
Dec 2013 - 17.0%
Dec 2013 - 16.8%
Dec 2013 - 16.4%
Jan 2014 - 14.4%
Jan 2014 - 14.2%
Jan 2014 - 14.2%
Jan 2014 - 15.2%
Feb 2014 - 15.0%
Feb 2014 - 15.5%
Feb 2014 - 15.5%
Feb 2014 - 13.7%
Feb 2014 - 13.3%
Feb 2014 - 14.2%
Mar 2014 - 14.2%
Mar 2014 - 14.5%
Mar 2014 - 14.5%
Mar 2014 - 14.7%
Mar 2014 - 13.8%
Mar 2014 - 13.0%
Mar 2014 - 12.5%
Apr 2014 - 12.5%
Apr 2014 - 12.7%
Apr 2014 - 12.7%
Apr 2014 - 12.3%
Apr 2014 - 11.4%

In case you're wondering, the final update introduces the Easter polls from ICM and Survation in one go. And it must be admitted that Survation's entry into proceedings in late January does distort the trend slightly, because they started out with an extremely No-friendly methodology in their first poll, and then introduced a methodology much closer to ICM and Panelbase from their second poll onwards.

Finally, here is Scott Hamilton's updated graph for the extrapolated trend after the Easter polls.  This uses a completely different method - it's based on a rolling average of the last eight polls to be published, rather than the most recent poll from each firm.

(Click to enlarge.)