Friday, October 23, 2015

Unveiling the Scot Goes Pop Poll of Polls for the EU referendum

I know this falls into the category of "he would say that, wouldn't he", but I always felt that the Poll of Polls I ran on this blog during the independence referendum campaign gave a much more meaningful indication of the direction of travel than John Curtice's Poll of Polls. The Curtice version was an average of the most recent few polls, regardless of which firms had conducted them, or what methodology had been used. That meant it sometimes gave the illusory impression that Yes or No were making progress simply because the newest update included a larger number of polls from Yes-friendly or No-friendly firms. By contrast, the Scot Goes Pop Poll of Polls only included the most recent poll from each firm, meaning that any statistically significant changes were more likely to reflect real movement on the ground.

Professor Curtice is sticking to his preferred method for his new EU referendum Poll of Polls, and if anything that's going to create even bigger problems this time, because the gulf between telephone and online polls is absolutely enormous at the moment. Telephone firms are suggesting there is a hefty lead for 'Remain', while online firms think it's either a statistical tie, or very close to that. Online polls are likely to appear more frequently, because they're cheaper to conduct, so any Poll of Polls that gives equal weight to all recent polls is automatically giving greater weight to the 'Leave'-friendly online data collection method. In fact, the current update of the Curtice PoP is almost entirely comprised of online polls, producing a result of Remain 54%, Leave 46% (with Don't Knows excluded). Unless there is some rational reason for concluding at this early stage that the online method is far more reliable (and there isn't), those figures don't meaningfully reflect the totality of current polling evidence. And because the exact weighting given to online polls will vary from update to update (sometimes it might rise to 100%, occasionally it might drop to 50%), future results will fluctuate in an almost random way.

So I thought it might be a good idea to once again run a Poll of Polls here that is designed to be better at picking up the trend. I'm not going to replicate the method I used for the independence referendum, because I think that might get too complicated - some firms will be regularly conducting both telephone and online polls, so I would have to decide whether to count "ICM (telephone)" and "ICM (online)" as separate firms. Instead, I'm going to produce three averages - one for all telephone polls conducted at least partly within the last month, one for all online polls conducted at least partly within the last month, and a composite 'headline' set of figures that give an equal 50/50 weighting to telephone and online. Obviously the latter average will be a bit artificial, but at least this way we'll get a sense of what the real state of play is if telephone polls are correct, or if online polls are correct, or if both types of poll are wrong by roughly an equal amount. Under the current circumstances, I'm not sure it's possible to do a whole lot better than that.

Should the United Kingdom remain a member of the European Union or leave the European Union?


Remain 48.7%
Leave 37.2%


Remain 43.8%
Leave 38.3%


Remain 53.5%
Leave 36.0%

(The online average is based on six polls - five from ICM and one from YouGov. The telephone average is based on one poll from ComRes and one from Ipsos-Mori.)

As big as the gap between the two component averages looks, it could easily have been much bigger - the new telephone poll from Ipsos-Mori caused a surprise by showing a slump in the 'Remain' lead. That can't be explained away simply by the introduction of the new question, because they also asked exactly the same question that was posed in the firm's previous poll in June, producing much the same result. It could be that the swing is being exaggerated due to normal sampling variation, but unless either this poll or the last one was an outright rogue, it seems likely that it's picking up some kind of genuine shift.


  1. I suspect there is a shift looking at how the EU is subverting democracy in some its members countries.

    OT: I thought this might be of interest to you polling experts.

  2. Glasgow Working ClassOctober 23, 2015 at 11:22 PM

    Junker is interferring in Portugal. Scottish Nat sis need to explain why they want to dump their neighbours and go for the corrupt EU. Maybe its the old bitter Jacobite blood.

    1. Have you been on the piss again,you insignificant trolling Tory tosser?

    2. Cereal for the Tory "working" class...

  3. I think most Scots would be content if the UK was like a smaller EU.
    A union of independent countries with a single market.

    As things stand, there is a huge difference between the two types of union.
    If the 'Leave' supporters think the EU is restrictive, imagine how they would feel if the UK parliament only had the same powers as Holyrood ..

  4. I must say that I welcome that fact that many seem to be willing to question what way they will go in voting for this.

    A while back, there was a tendency (still in operation, though seemingly not quite as strongly correlated) towards seeing it through the prism of how one feels about Scots indy (i.e. 'I voted Yes, therefore I will vote to Remain). I would be very interested in seeing how much slippage there has been in that attitude and hopefully we will get a sense of it from the poll Wings is going to undertake.

    For myself, I'd like to think we will get a bit of an education between now and the vote as I am on the fence. I'll not count on receiving one, though. It is clear being in the EU has offered some benefits, but it also seems to be abusing its sway so it is not an easy decision at all. I'd also like us to start thinking in terms of how, were we to leave, it would be possible to make sure we aped some of the good things the EU has done anyway, Obviously this would be a lot easier in an indy Scotland, but who says we cannot just have laws which mimic the best stuff without all the kowtowing to bankers and bullying other states?

    1. You raise some interesting points, and reading it, 'the good things the EU has brought' made me think about how if we left, the Tories could annul EU regulations, for instance, on reducing landfill. The fact that we would no longer need to remain compliant with such EU rules as we think are progressive (for instance on employment law are the environment) is a very good reason for staying.

  5. Are these polls UK wide or Scotland specific .