Introducing the Scot Goes Pop Poll of Polls

Well, I say "introducing" - this may be its one and only outing if I can't be Smithsoned to keep it going. But after the appearance yesterday of a highly dubious poll, I thought now might be a good moment to seek a more realistic assessment of the current state of play, based on the widely varying figures from all the credible pollsters. Now, I'll repeat what I've said in the past - there's no reason to automatically assume that an average of the polls will produce a more accurate result than the findings of any individual pollster. If the majority of pollsters are making methodological mistakes that are all tending in one direction, then that would distort the average, and it could well be that (for example) the more rosy figures for the pro-independence campaign produced by Angus Reid and Panelbase are closer to the truth than the overall average. But the problem is that, prior to polling day, we simply have no way of knowing which pollster (if any) has its methodology right, so there's at least a case to be made that a polling average is likely to be less inaccurate than any other method we could dream up for attempting to divine the true position.

Here are the principles that the Poll of Polls will be based on -

1) Only polls conducted by British Polling Council members will be included in the sample - as of now, that means Panelbase, Ipsos-Mori, Angus Reid, ICM, YouGov and TNS-BMRB. I see no case whatever for including polls conducted by companies that don't even bother to match the BPC rules for transparency. As I said yesterday, I simply cannot fathom why Professor John Curtice is treating the Progressive Scottish Opinion poll in the same way as all the others. Given that he felt able to airily dismiss the partial information from internal Yes campaign polls due to a lack of transparency, it's incredibly hard to understand how he can justify his reverential treatment of a pollster that does not reveal its methodology, datasets, or even something as basic as the question it asked respondents. And as Progressive Scottish Opinion have been commissioned by a rabidly anti-independence newspaper, the latter point in particular ought to be setting alarm bells ringing.

2) I will not be playing Kellner-type games by discriminating between different BPC members on the basis of whether I happen to agree with their numbers. All BPC members (including Ipsos-Mori, who are currently the least favourable for the pro-independence campaign) will be treated absolutely equally.

3) To ensure this equality of treatment, I will not be following the traditional Poll of Polls method of including all polls conducted within a certain time-frame. Instead, I will simply use the most recent figures from each pollster, regardless of how long ago that poll was conducted. In case any passing No supporter is sceptical about my reasons for doing this, let me emphasise that this method will actually significantly reduce the percentage for Yes, and increase the percentage for No, because there has been a disproportionate number of Panelbase polls recently, and only the most up-to-date one will be taken into account. But by doing it this way any changes we see over time will be meaningful, rather than an artificial product of differing levels of activity from different pollsters.

One other point worth making is that the whole sample comes from polls that were conducted before the publication of the White Paper. So this will give us a useful baseline from which to judge the impact of Scotland's Future.

MEAN AVERAGE (not excluding Don't Knows) :

Yes 32.0%
No 49.5%

MEAN AVERAGE (excluding Don't Knows) :

Yes 39.3%
No 60.7%

MEDIAN AVERAGE (excluding Don't Knows) :

Yes 38.8%
No 61.2%

The observant among you will already have noticed a few things -

1) When Don't Knows are included, the anti-independence vote is fractionally below the pyschologically-important 50% threshold.

2) The average figures are actually somewhat closer to Panelbase on the "Yes-friendly" end of the spectrum than they are to Ipsos-Mori on the "No-friendly" end. This completely blows out of the water Alex Massie's suggestion of a few months ago that it's somehow "Panelbase versus the field".

3) The swing required for Yes to win from the position implied by the Poll of Polls is considerably less than the SNP achieved over a much shorter timescale prior to their 2011 Holyrood win. In January 2011, a TNS-BMRB poll showed Labour ahead by 16%. By polling day in May, the SNP had turned that around by force of argument to a 14% SNP lead - a swing of 15% over just four months. By contrast, if this Poll of Polls is accurate, the Yes campaign would require a swing of just 8.75% over the next nine-and-a-half months to draw level (or a 10.7% swing if Don't Knows are excluded).

In case you're wondering, it's not really feasible to produce a median figure that doesn't exclude Don't Knows - it would produce too many statistical anomalies. For instance, the median lead for No could easily be totally different to the gap between the median Yes figure and the median No figure.