New Panelbase referendum poll

There's a new Panelbase poll out tonight on referendum voting intentions, which as you'd expect shows a more favourable picture for Yes than most other recent polls - although, perhaps slightly surprisingly, the "gold standard" ICM remains in harness for the moment as the best pollster for the pro-independence campaign.

Should Scotland be an independent country?

Yes 37% (-1)
No 49% (+2)

The trend figures are on the face of it at odds with what we've seen recently from every other pollster, although it has to be borne in mind that John Curtice pointed out last time around that Panelbase had made a significant methodological change to correct for the fact that they had previously underrepresented older voters.  If I recall correctly, Curtice implied that such a change ought to have artificially generated an increase in the No lead of up to 4%, but in fact the increase was just 1%.  It may be that margin-of-error effects flattered Yes in the last poll, and that as a result the change is only now making itself felt in the headline numbers - in other words, the apparent slight increase in the No lead may be a mirage.

As of this moment, Curtice hasn't yet posted an analysis of this poll on his blog and the datasets aren't available on the Panelbase website, so it remains to be seen if there are any other new contributing factors.  One revelation about Panelbase's methodology that I did find rather troubling, though, came from someone who took part in one of their polls about ten days ago - it may well have been this very poll, although I haven't seen the fieldwork dates yet.  In case you didn't see what Calum Findlay had to say at the time, here's the exchange in full -

Calum Findlay : We should expect a new Panelbase poll in the next few weeks, I've just taken one.

I've taken part in their surveys before, and I thought you might be interested to know that they always use a preamble they don't show in their results tables:

"As you may know, the Scottish government intends to hold a referendum this year on Scotland becoming a country independent from the rest of the United Kingdom. The question on the ballot is expected to be as below. How would you vote in this referendum?

Should Scotland be an independent country?"

Me : Hi Calum, that really is appalling, in two ways - firstly, because the wording is tortured and potentially leading, and secondly, because they don't own up to it.

Are you sure that they've always done it? Because if (in the nightmare scenario) this is a new wording it might affect the results quite significantly. We know what happened when YouGov finally dropped the leading words from their own preamble last September.

Calum Findlay : This is maybe my 4th survey on independence over the last year, and I am 100% that they have always used that preamble.

The wording of the question isn't that big of a surprise considering that the Sunday Times commissioned it, but the fact you would never have known unless you had taken the survey is strange.

I'd say the exact same about Ipsos as well. They mention in their summaries they ask respondents how they would vote tomorrow (I remember a man from Ipsos even saying that on STV News), but no sign of a preamble has ever been in their results tables. Judging by the No leads they manage to produce, there's a chance that it is very leading indeed.

If Calum is right that the 'stealth' preamble has been in use for some time, then it obviously wouldn't be responsible for the trend in this poll, but the possibility can't be excluded that it's been harming the Yes vote all along in the headline numbers - and that is where the anti-independence agenda of Panelbase's client is bound to raise a few concerns. It seems fairly clear from the datasets that the preamble wasn't used in the SNP-commissioned Panelbase poll a few months ago that produced an outright Yes lead. At the time, commentators rubbished that poll on the grounds that the sequence of questions had probably generated a freak result, but in retrospect it's just as conceivable that a different preamble was producing a different answer from some respondents. Was the more usual preamble originally put in place at the bidding (or under the influence) of the Sunday Times, or was it genuinely Panelbase's own decision? We simply don't know, but when a pollster proves to be unnecessarily secretive about a point like this, suspicions are bound to arise.

One small point I would make in Panelbase's favour, though, is that at least they don't ask respondents to imagine that they are taking part in a hypothetical referendum "tomorrow".  The same is true of ICM and TNS-BMRB, and it may well be no coincidence that all three pollsters are showing a significantly lower No lead than either YouGov or Ipsos-Mori, both of whom insist on the "tomorrow" qualifier.  The point being, of course, that some voters may be thinking to themselves : "I would have to vote No if I was rushed into a decision tomorrow without having heard all the arguments, but if I'm reassured by what I hear I may vote Yes in September."

* * *

So what can we say about the trend between the late summer/early autumn of 2013 and early 2014? We can now compare the findings of four different BPC pollsters, and to put it mildly the pictures they paint are varied...

ICM are suggesting a big swing of 5% to the Yes campaign since September. There is some reason to believe that the swing may have been slightly exaggerated due to sampling issues relating to young respondents, but even allowing for that the movement looks substantial.

TNS-BMRB are also suggesting a big swing of 4.5% to the Yes campaign since late August/early September. Again, there is some reason to believe that this may be a slight exaggeration due to clear methodological mistakes in the Aug/Sept poll, but those were rectified in the late September/early October poll, and even since then there has been a significant swing of 3% to Yes. A gradual month-on-month decline in the No lead has bolstered the impression that the swing is genuine and potentially ongoing.

YouGov have shown a huge swing of 6% to Yes since their August poll which used their old methodology, but just a 1% swing to Yes since their September poll which used a vastly improved methodology. The latter swing would be well within the margin of error, whereas the former would obviously not be.

Panelbase are showing a tiny swing of 1% to No since their late August/early September poll, which is well within the margin of error. Taking into account the recent No-friendly methodological changes mentioned earlier in this post, the position can essentially be regarded as unchanged, leaving Panelbase as the only remaining pollster showing a picture of seemingly relentless stability.

So the situation is as clear as mud, but what you can't do is just cherry-pick the pollster whose story suits you best (which with supreme irony the No campaign are suddenly trying to do with Panelbase!). Taking the four together, it seems highly unlikely that the small swing to No reported by the new Panelbase poll is real, but equally it also now seems unlikely that the swing to Yes is quite as high as ICM suggested. The truth probably lies somewhere between the two extremes, with a small-to-middling movement towards Yes, perhaps amounting to a swing of between 1% and 4% over the last five months. To put that into perspective, in mid-September an average of the polls suggested that Yes required a 9% swing to draw level, so anything between one-ninth and just under one-half of that may have already been achieved. The Poll of Polls figures given below would be bound to lag behind any movement on the higher end of that scale, because they include one poll that is several months old, and have also been distorted by the recent intervention of Survation.

The next indication may be provided by TNS-BMRB, who are apparently planning to release another poll in mid-February (ie. in a few days).

*  *  *

SCOT GOES POP POLL OF POLLS

This is the ninth update of the Poll of Polls, and only the second to show a small swing towards No.  Six of the previous updates have shown pro-Yes swings, with one showing an unchanged position.

MEAN AVERAGE (not excluding Don't Knows) :

Yes 33.9% (-0.1)
No 49.0% (+0.3)

MEAN AVERAGE (excluding Don't Knows) :

Yes 40.9% (-0.2)
No 59.1% (+0.2)

MEDIAN AVERAGE (excluding Don't Knows) :

Yes 40.8% (n/c)
No 59.2% (n/c)

(The Poll of Polls is based on a rolling average of the most recent poll from each of the pollsters that have been active in the referendum campaign, and that adhere to British Polling Council rules. At present, there are seven - Angus Reid, YouGov, TNS-BMRB, Survation, Panelbase, Ipsos-Mori and ICM. Whenever a new poll is published, it replaces the last poll from the same company in the sample. Changes in the Poll of Polls are generally glacial in nature due to the fact that only a small portion of the sample is updated each time.)

One interesting quirk caused by Panelbase's marginal move "back towards the pack", as it were, is that we are now closer to convergence between the mean and median averages than ever before.

The pro-independence campaign now require a swing of 7.6% on the headline average to take the lead.

* * *

I see that Blair McDougall, the No camp's embarrassment of a campaign chief, is crowing about the Panelbase poll on Twitter, which I must say is a rather odd thing for him to do. If he doesn't want to make himself look even more of a fool for his bizarre lie on Scotland Tonight the other week that "every poll has shown Yes at between 25% and 33%", I'd have thought he'd want to keep very, very quiet about a poll showing the Yes vote falling to 37%! (And it's as high as 43% with undecideds excluded...)