Sunday, May 18, 2014

Panelbase confirms referendum remains on a knife-edge as No campaign lead by just 7%

In perhaps the most misjudged comment of the day so far (although there's still plenty of time), Mike Smithson of Political Betting reacted to the ICM poll by declaring "panic over", and by giving our masters in Westminster his personal blessing to forget all about the Jockland distraction and get on with the much more serious business of the European elections.  Just one snag, though - he was apparently oblivious to the fact that a Panelbase poll was published at the same time as ICM, which showed the No campaign with a lead of just 7%.  That's somehow not quite what I picture when I try to imagine what the end of panic at McDougall Central would look like.

Should Scotland be an independent country?

Yes 40% (n/c)
No 47% (+2)

I don't yet know what the figures are with Don't Knows excluded, but a rough calculation suggests it's most likely to be Yes 46% (-1), No 54% (+1).

In combination with the Survation poll of a few days ago, this is a very strong pointer (albeit not quite absolute proof) that the rise in the No lead seen in the ICM poll is an artifact of the margin of error, rather than something real.  The fieldwork for the three polls will have been conducted at roughly the same time, and it's fantastically improbable that both Panelbase and Survation would have failed to pick up the trend shown by ICM if it actually reflected the reality on the ground.  The recent poll from TNS-BMRB also failed to replicate the ICM trend (quite the reverse - it showed the No lead at its lowest level of the campaign so far), although admittedly that's of less help because the fieldwork is slightly out-of-date.

Taking all of the available evidence together, there are only really two realistic possibilities - a) there has been no movement at all recently, or b) there has been a very small increase in the No lead in the last two or three weeks, probably no bigger than 1% or 2%.  The only reason for entertaining the latter possibility is that this poll breaks a sequence of three Panelbase polls in a row to show a No lead of exactly 5% - but of course even if public opinion had remained absolutely static, it was statistically impossible for that sequence to continue due to the margin of error.  It may just be random chance that this poll has a slightly higher lead rather than a slightly lower lead.  [UPDATE, 4pm : The unrounded figures from Panelbase show that the raw Yes vote has actually increased, and that the No lead has only increased by 0.8%, rather than the 2% misleadingly suggested by the rounded percentages.  That makes it even more likely that the changes are just margin of error noise, and that the ICM poll is an outlier in terms of the trend it shows.  Future polls may prove me wrong, but at the moment my best guess is that there has been no movement back towards No whatsoever.]

That said, one of the fascinations of this poll is that it's the first Panelbase poll in several months to have been commissioned by an anti-independence client.  All of the firm's previous polls for the Sunday Times used a subtly biased (and for a long time highly secret) preamble to the referendum question, whereas all of their recent polls for Yes-supporting clients have used a scrupulously neutral preamble.  So in order to interpret the result of the new poll correctly, it's vitally important to take account of which preamble was used - something which may not became clear until the datasets are published.  If it was the old preamble, then this poll is not directly comparable with the ones showing a 5-point lead, and the most recent poll it can be properly compared to is the one in early February showing a 12-point lead.  But let's hope common sense has prevailed, and that Panelbase have decided to use the neutral preamble from now on regardless of client. [UPDATE : I was a bit troubled that the Panelbase datasets do not reveal the preamble used, but Calum Findlay took part in the poll and has confirmed the neutral preamble was used.]

Either way, though, "Better Together" have repeatedly made clear that they refuse to acknowledge the existence of Panelbase polls conducted by the SNP, Newsnet Scotland, Wings Over Scotland and Yes Scotland, so we can all look forward to McDougall and co mourning the fact that their lead has just dropped a hefty 5% since the last "valid" Panelbase poll.  Unless of course they're about to move the goalposts yet again.  But I can't believe they'd do that - these guys have integrity.

Perhaps the most important point that needs to be made is that the No lead is still below Panelbase's previous normal range of 8-13% - which remained in place as recently as late February, long after other pollsters had started showing a post-White Paper surge for Yes.  So if Panelbase are to be believed, Yes have actually made net progress since the end of winter - a fact which inconveniently refuses to tally with the Gospel according to Professor Curtice.

This is the first poll in which Panelbase have sought a breakdown of voting intention by country of birth, and that has turned up an uncannily similar picture to the one in last month's ICM poll.  Respondents who were born in Scotland favour independence by a 2% margin, while a surprisingly hefty minority of English-born respondents (more than a quarter) are also now in the Yes column.  However, the No vote among English people is somewhat higher than ICM reported.  But of course the most important thing to emerge from the ICM poll was that the sampling was all wrong, and that English people were being significantly over-represented, thus artificially increasing the No lead.  It'll be fascinating to see whether Panelbase's sample is more representative (or indeed has been specifically weighted by country of birth).

*  *  *


Obviously two polls overnight showing an increase in the No lead was always bound to feed into this update of the Polls of Polls, but apart from the fact that these changes may be illusions caused by the margin of error, the other crucial caveat that needs to be borne in mind is that Ipsos-Mori are still represented in the sample by a poll conducted in late February that gave No a handsome lead, whereas there are very strong indications that the firm's more recent secret mega-poll for the UK government showed a much more favourable position for Yes.

As an aside, this is one of those moments that shows up the flaws in the method used by the ScotCen and Financial Times Polls of Polls. The Yes vote in the ScotCen figures (which excludes Don't Knows) currently stands at 43%. The average Yes vote in these two new polls is roughly 44%. So ScotCen may well end up reporting a decreased No lead off the back of two polls that actually show an increased lead!  [UPDATE, 4pm : That isn't how it worked out, but presumably only because favourable polls for Yes just happened to drop off the other end in the rolling average.]

MEAN AVERAGE (excluding Don't Knows) :

Yes 42.0% (-1.1)
No 58.0% (+1.1)

MEAN AVERAGE (not excluding Don't Knows) :

Yes 35.0% (-0.8)
No 48.3% (+1.0)

MEDIAN AVERAGE (excluding Don't Knows) :

Yes 42.3% (-0.7)
No 57.7% (+0.7)

(The Poll of Polls is based on a rolling average of the most recent poll from each of the pollsters that have been active in the referendum campaign since September 2013, and that adhere to British Polling Council rules. At present, there are six - YouGov, TNS-BMRB, Survation, Panelbase, Ipsos-Mori and ICM. Whenever a new poll is published, it replaces the last poll from the same company in the sample. Changes in the Poll of Polls are generally glacial in nature due to the fact that only a small portion of the sample is updated each time.)

Here are the long-term trend figures, with the updates prior to Easter recalculated to exclude the inactive pollster Angus Reid...

The No campaign's lead in the Poll of Polls mean average (not excluding Don't Knows) :

Sep 2013 - 21.6%
Sep 2013 - 21.4%
Sep 2013 - 19.4%
Oct 2013 - 18.8%
Oct 2013 - 18.4%
Oct 2013 - 18.2%
Nov 2013 - 18.4%
Nov 2013 - 18.0%
Dec 2013 - 17.0%
Dec 2013 - 16.8%
Dec 2013 - 16.4%
Jan 2014 - 14.4%
Jan 2014 - 14.2%
Jan 2014 - 14.2%
Jan 2014 - 15.2%
Feb 2014 - 15.0%
Feb 2014 - 15.5%
Feb 2014 - 15.5%
Feb 2014 - 13.7%
Feb 2014 - 13.3%
Feb 2014 - 14.2%
Mar 2014 - 14.2%
Mar 2014 - 14.5%
Mar 2014 - 14.5%
Mar 2014 - 14.7%
Mar 2014 - 13.8%
Mar 2014 - 13.0%
Mar 2014 - 12.5%
Apr 2014 - 12.5%
Apr 2014 - 12.7%
Apr 2014 - 12.7%
Apr 2014 - 12.3%
Apr 2014 - 11.4%
May 2014 - 11.2%
May 2014 - 11.2%
May 2014 - 11.5%
May 2014 - 13.3%


  1. Nice spin on a lost cause

  2. With 122 days to go to the 2011 election now, current polls have this:

    45% Labour
    32% SNP
    11% Con
    9% Lib

    Little point putting any money on Labour winning; you’ll not get really any return on it. Iain Gray is ‘guaranteed’ for FM. I mean we’ve just had a TNS poll with Labour on 49%!

  3. Panelbase showing a nice rise for Yes and gap closure.

    Yes = 40.0 (+2.1)%
    No = 44.5 (+0.6)%

    From the last times poll in February.
    47 (+5)%
    53 (-5)%

    Since the last times poll in February. Also up on the WoS April where full base data was shown (YesScotland poll didn’t have this).

    Note this is before turnout weighting which I’m uncomfortable with as it takes people’s word on this.

    Given that up to a 1/4 of respondents regularly lie in polls and say they voted in 2011 when they didn’t, how trusting can we be here.

  4. Panelbase looking to be the most demographically representative pollster.

    This one has 79.2% born in Scotland among respondents. Age 16+ in 2011 census was 81.5%.

    ICM are further off. Their last poll had only 75% of respondents born in Scotland; a 6.5% bias to people from England/the rUK giving No an advantage.

  5. Anon : "Nice spin on a lost cause"

    Give me a break, I didn't say a word about Ed Miliband.

  6. Damn, excel error. Ignore above Figures

    From the last Times poll in February (ex DK):

    46% (+3) Yes
    54% (-3) No

    For full base.

    What's particularly interesting is how little weighting panelbase need. Hardly anything needed by age group, past vote etc.

    There's a bit for ABC1/C2DE, but even that makes little difference to the unweighted numbers in terms of Y/N.

    Agrees with Panelbase getting the CoB data nearly bang on without weighting in that their panel is the most demographically representative of Scotland.

    Probably because they are based in / started out from the NE of England polling that area and Scotland.

  7. I took the Panelbase poll, it asked "How do you intend to vote in response to the question: Should Scotland be an indepdence country?"

  8. Thanks, Calum, that's a relief. I don't really understand why they're keeping that secret, unless it's an oversight.

  9. One interesting thing about the poll was it also asked if Salmond's Putin comments had made my opinion of him more positive, negative or if it hadn't made a difference.

    I notice the results of that question haven't been published.

  10. Calum - did the poll mention when you would be voting on indy, i.e. today / tomorrow / September / or no date mentioned?

  11. It asked the exact same question as TNS word for word:

    "There will be a referendum on Scottish independence on the 18th September 2014. How do you intend to vote in response to the question: Should Scotland be an independent country?"