The new Survation poll for the Daily Record has just been released, and it shows a strikingly similar picture to Sunday's TNS poll, which had the Yes vote at its highest level of the campaign so far, but also recorded an increase in the No vote as undecideds were squeezed. The similarity between the two polls will of course be masked by the fact that TNS are a No-friendly pollster, and even with them showing Yes at a new high watermark, that still only amounted to 41.4% after undecideds were excluded. By contrast, Survation are one of the more Yes-friendly pollsters, and tonight they're showing the pro-independence campaign on the brink of victory -
Should Scotland be an independent country?
Yes 41% (+2)
No 46% (+2)
With Don't Knows removed, it works out as...
Yes 47% (n/c)
No 53% (n/c)
Although the latter set of figures appear to be unchanged, it's worth remembering that prior to rounding Yes were actually on 46.6% last month. So unless the rounding has flattered Yes again, it seems highly likely that they've reached a new record high with Survation even after undecideds are stripped out. We'll find out for sure when the datasets are released, presumably tomorrow.
I think this could be one of the most important polls of the campaign to date, and I'll explain why. In spite of the similarity to the TNS trend, a key difference is that TNS have shown Yes hovering at the same record high level (after Don't Knows are excluded) for several months now, without making any further significant progress, and without suffering any reverses. Survation on the other hand only put Yes at 47% for the first time last month. As I noted at the time, I was slightly concerned that poll might prove to be an outlier, because alarm bells were ringing in my head due to odd patterns in the datasets. Time well tell what the datasets look like this time, but it's dramatically less likely that Survation would produce two very similar 'rogue polls' in consecutive months. So it seems reasonable to conclude that these numbers are probably being caused by a real increase in support for Yes, which is most likely to have occurred at some point over the last two months.
That's important, because the picture painted by the previous six polls was as clear as mud, and until tonight it was scarily unclear which way the wind was blowing. We had Panelbase, Survation and TNS all showing Yes at a high watermark for the campaign - albeit only by the tiniest of fractions in the case of TNS. We had ICM showing a decrease in the No lead, but that was only a reversion to the previous 'norm'. We then had YouGov with two successive polls that showed small increases in the No lead - and crucially the fieldwork for the second of those polls was more recent than all of the other polls. So until tonight, there was the tantalising possibility for the No campaign that YouGov might not merely have punctured the sense that Yes had reached an all-time high, but could actually be showing a new and potentially decisive trend in the opposite direction. Well, that theory has just gone out of the window, because whatever Survation's fieldwork dates were, they'll certainly have been much more recent than YouGov's.
Admittedly, it's still very hard to reconcile the trend reported by YouGov with what we've been seeing from elsewhere. We can now very safely assume that the apparent increase in the No lead was just margin of error 'noise', but if Panelbase and Survation are right, you'd have expected YouGov to show a further boost for Yes, not merely a static position disguised by noise. So there's still at least a small degree of uncertainty about the underlying trends, and it may take two or three more polls for the mists to entirely clear.
The biggest effect of the new Survation poll will of course be psychological. It's just been bad luck, but the last three polls to be published were all from No-friendly firms, and that allowed the false impression to take root in a few sections of the media that there was some kind of polling consensus of a significant No lead. Tonight's numbers will very abruptly break that spell. (Except maybe in those publications which always report "ANOTHER blow for Salmond" regardless of what the numbers show!)
* * *
SCOT GOES POP POLL OF POLLS
Tonight's update of the Poll of Polls sees the No lead hold steady at 11% when Don't Knows are taken into account - but that remains lower than any No lead seen prior to a few weeks ago, and is barely half of the lead recorded back in September. Meanwhile, after undecideds are stripped out, the gap has slightly narrowed again.
MEAN AVERAGE (excluding Don't Knows) :
Yes 43.6% (+0.1)
No 56.4% (-0.1)
MEAN AVERAGE (not excluding Don't Knows) :
Yes 37.2% (+0.4)
No 48.2% (+0.4)
MEDIAN AVERAGE (excluding Don't Knows) :
Yes 43.3% (n/c)
No 56.7% (n/c)
(The Poll of Polls is based on a rolling average of the most recent poll from each of the pollsters that have been active in the referendum campaign since September 2013, and that adhere to British Polling Council rules. At present, there are six - YouGov, TNS-BMRB, Survation, Panelbase, Ipsos-Mori and ICM. Whenever a new poll is published, it replaces the last poll from the same company in the sample. Changes in the Poll of Polls are generally glacial in nature due to the fact that only a small portion of the sample is updated each time.)
Here are the long-term trend figures, with updates prior to Easter recalculated to remove the inactive pollster Angus Reid ...
The No campaign's lead in the Poll of Polls mean average (not excluding Don't Knows) :
Sep 2013 - 21.6%
Sep 2013 - 21.4%
Sep 2013 - 19.4%
Oct 2013 - 18.8%
Oct 2013 - 18.4%
Oct 2013 - 18.2%
Nov 2013 - 18.4%
Nov 2013 - 18.0%
Dec 2013 - 17.0%
Dec 2013 - 16.8%
Dec 2013 - 16.4%
Jan 2014 - 14.4%
Jan 2014 - 14.2%
Jan 2014 - 14.2%
Jan 2014 - 15.2%
Feb 2014 - 15.0%
Feb 2014 - 15.5%
Feb 2014 - 15.5%
Feb 2014 - 13.7%
Feb 2014 - 13.3%
Feb 2014 - 14.2%
Mar 2014 - 14.2%
Mar 2014 - 14.5%
Mar 2014 - 14.5%
Mar 2014 - 14.7%
Mar 2014 - 13.8%
Mar 2014 - 13.0%
Mar 2014 - 12.5%
Apr 2014 - 12.5%
Apr 2014 - 12.7%
Apr 2014 - 12.7%
Apr 2014 - 12.3%
Apr 2014 - 11.4%
May 2014 - 11.2%
May 2014 - 11.2%
May 2014 - 11.5%
May 2014 - 13.3%
Jun 2014 - 12.1%
Jun 2014 - 12.1%
Jun 2014 - 11.3%
Jun 2014 - 9.9%
Jun 2014 - 10.3%
Jun 2014 - 10.7%
Jul 2014 - 11.0%
Jul 2014 - 11.0%
Both went up the same, so surely no big deal.
ReplyDeleteHowever, weeks of fearmongering and the YES vote is solid.
Although our vote is increasing its a concern to find the No vote also increasing.
ReplyDeleteGood poll though of course the amusing out of touch twits are scrambling around desperately looking for excuses.
ReplyDeleteThe usual drivel about 'no progress' is being parroted by Curtice and his clueless tory fanclub. I mean it's not as if there was a massive and very quick turn around in the polls in 2011 and the referendum is next monday, isn't it?
*chortle*
This would also be the same Curtice who was left looking like an idiot after Survation were forced to defend themselves against Kellner's bizarre attack and his preposterous YouGov "Keller Correction" which has been rightly laughed at for it's many idiotic assumptions. Not by Curtice though. Curtice was happy to get quoted endorsing Kellner and YouGov's poll while completely ignoring the absurd "Kellner Correction".
Curtice is also parroting the desperate tory and 'better together' spin about division and "broken relationships" which is a frankly feeble attempt to try and stir up trouble and smear a democratic referendum as something unpleasant.
I note with vast amusement that oldnat set Curtice straight on his own blog as well as the usual witless shrieking from the scottish tory sugers and out of touch twits on 'stormfront lite' politicalbetting.
LOL
That's also well over a week since Kellner was asked perfectly reasonable questions about YouGov's bizarre methodology and yet still no answers. Pretty damn telling I would say.
So alongside TNS, it would appear that the DKs are starting to break, but at best it's 50/50 - not great news, is it?
ReplyDeleteSo, we continue to have a steadily rising Yes but measurably more volatile No as people switch between saying No, DK and even Yes sometimes.
ReplyDeleteThis implies a solid Yes base building with new recruits in addition to shy Yes coming clean. In contrast, there's weakness in No due to some on the No side not being quite what they seem.
Women show the higher level of No volatility and this was particularly evident in ICM and their 'feeling uncomfortable about your VI is something that you should be feeling otherwise we wouldn't ask' poll. Looking back:
ICM Jan 2012
Men
41% Y
47% N
Women
39% Y
40% N
Now that's a little odd huh?
Must be a rogue.
TNS face to face August 2011
Men
43% Y
40% N
Women
36% Y
36% N
Oh.
Of course these were done when it was more acceptable to be Yes / while the anti-Scotland propaganda machine was still in low gear.
Lots of other examples.
Strangely, Survation just started following me on Twitter. I don't even say that much (except that I keep asking Jim Murphy how he managed to get 100 days off his work.
ReplyDeleteRobert
"there's weakness in No due to some on the No side not being quite what they seem."
ReplyDeleteFrom doorsteps and meetings I'd agree with this totally. Once people are saying Yes, I find they generally don't switch back. And there is a movement among my FB friends going from no, to silent, to undecided, to tentative yes then with a few onto active yes. Over the final month I suspect a huge amount of "silent yes" people will start "coming out".
By contract on the no side there are very few really hard, campaigning, active no voters. I know a few who are very definite emotional no voters and they won't change, but they're not out campaigning. And the really hard, campaigning no voters, like the OO probably do more harm to the cause than good.
But the usual no voters tend to be either people who simply haven't given it much thought at all and aren't engaged, or who are "heart yes" but "head no" because they get all their information from the BBC and mainstream media, and that has been so relentlessly negative. Often those types of no need very little to shift them to yes because they just want to be re-assured.
A public meeting will often flip them to yes, then links to further information allows them to go off and discover for themselves their own reasons why they're yet.
Add to that large numbers of alienated people registering for the first time who may not be being picked up by polls, and you can see grounds for a pretty hefty potential shift in the final weeks.
But the usual no voters tend to be either people who simply haven't given it much thought at all and aren't engaged, or who are "heart yes" but "head no" because they get all their information from the BBC and mainstream media, and that has been so relentlessly negative. Often those types of no need very little to shift them to yes because they just want to be re-assured.
ReplyDeleteSure. That's why e.g. the prospect of the Tories again is all that's required to cause a sudden shift and put Yes in front. And we will get the Tories again, likely next year.
As I keep saying, if there's a No majority, we'll get the Tories as sure as night follows day. Cameron is already positioning himself as the master strategist who "saved the union". English voters are going to reward that handsomely.
ReplyDeleteThere is an up and down side I have found when out canvassing. When I talk to not sures and some Noes, I find they are generally a bit scared, but they are not arguing for the Union, as the Together would like to portray it. That gives me hope that many more could be convinced to go for Yes.
ReplyDeleteOf course it comes down to numbers on the day; still I believe there will be a lot of desperately disappointed Noes when they realise they don't know what it was they voted No for.
If people think the polls are wrong, or there will be an eleventh-hour mass switcheroo by the voters, why spend so much time analysing the polls? You might as well look at chicken entrails.
ReplyDeleteThis is the rub. Posters can't say some polls are right when they show "yes is catching up", and then say others are wrong when it's apparent a becalmed set of figures isn't going to be enough to catch up in another 10 weeks.
That's not analysis, that's just a classic display of confirmation bias.
Survation just followed me on Twitter too. Maybe they're having a mass following day.
ReplyDeleteSorry if this is a point raised before but I keep reading that there's going to be an astonishingly high turnout. Over 80% or at least hight 70%s. How good are the polls at predicting turnout and do they tend to over- or under-estimate it? Intuitively, I'd expect fewer people actually to vote than tell a pollster they will but I've no idea if there's an observable rule to that effect and a standard margin of error. And might there not be a tendency, particularly here when everyone is telling everyone else that this is the most important decision in 300 years, to overstate your own likelihood to vote, even more so than in a general election? In the context of normal party politics it's become perfectly acceptable to adopt a "plague on all your houses" stance. Perhaps much less comfortable a thing to do in the context of the referendum? Not sure what effect it would have if any (and differential turnout's a slightly different point). Just struck me that it might be another reason why it's especially difficult for the polls to cope with the peculiarities of this campaign.
ReplyDeleteAnd as regards the point about "confirmation bias", I think the point is the marked consistent differences between different polling companies. People aren't picking and choosing different results from the same pollster. They're discussing which methodology is the most likely to give an accurate forecast, I think.
If people think the polls are wrong, or there will be an eleventh-hour mass switcheroo by the voters, why spend so much time analysing the polls?
ReplyDeleteWho is saying the polls are wrong?
Not me certainly.
Labour's huge lead going into 2011 wasn't a mistake by pollsters at all. The polls were not wrong; that's what people were saying to them.
Analysing polls is about working out what people are going to finally do based on what people are telling pollsters. They don't always tell the truth is the problem. Obviously there are some issues you methodology you can question and try to work out what effect this has.
At the moment, the union is toast in every poll bar the one where you put people on the spot; VI.
Was the same in 2011 right up to the last few weeks. If you had based your 2011 prediction on everything but VI, you'd have called a solid SNP win / possible majority from 2009. Gov sat, sat with Salmond, desire for a referendum on indy, dislike of the Tories, dissat with Labour, wish for devo max... It was all there. Everything said 'SNP big win and much bigger than last time'. Everything except VI until the last few weeks.
The situation was not too dissimilar ahead 2007, just on a smaller scale with the final shift less sudden so not as obvious.
History may or may not be repeating itself. In time we'll see.
Looks like the +2 for No is simply down to turnout filtering which I'm not totally comfortable with given this is aspect quite uncertain.
ReplyDeleteOn the full weighted base its:
38.9(+2.9)% Yes
42.0(nc)% No
Ex DK
48.1(+1.9)% Yes
51.9(-1.9)% No
That is an all time high for Yes and a repeat all time low for No from Survation.
You've done it just there, Scottish Skier. You've hunted through the raw data for something that give yes a higher share of the vote and then went "a-ha, i like this better". It is confirmation bias!
ReplyDeleteTo my mind excluding the don't knows is the bigger problem, as that's conflating "don't know" with "won't vote" which is wrong.
DK's in 1997 broke almost exactly along the same lines as everyone else, so excluding them isn't a crazy assumption tbh.
ReplyDeleteBut I agree about the hunting for a higher yes vote, both James and Scottish Skier tend to do it at times. But hey, it keeps the morale up, and its not like they both aren't though in their other analysis.
Agree that we need to be careful with confirmation bias, very easy to find what you want in any set if data.
ReplyDelete"a-ha, i like this better". It is confirmation bias!
ReplyDeleteLOL.
Speaks the person who is both seeking yet at the same time displaying confirmation bias.
I have always sought out data before turnout filters for the simple reason turnout is where pollsters have the worst record. It also yields a better value for 'mood of the nation' / what people are saying to each other potentially.
It's not the fault of pollsters; people lie in huge droves about turnout. They say they'll vote then they don't. Ask them if they voted and they'll say they did when they didn't. The shy turnout voter problem is as old as the hills. For the referendum, we don't even have previous ones to go on to gauge how much that might affect things.
69% of people gave a 2011 past vote to Survation. What amazing honesty. Of course in all probability the liars are saying what they would have voted if they had voted. But they're still lying.
Incidentally, turnout filters normally benefit No as Yes on balance seem more motivated. So I'm just giving No a better chance. As you will note, my 39% Yes is lower than Survation's 41%. So, I produce a lower Yes share and still those on the other side complain. Jeez.
;-)
incidentally, turnout filters normally benefit No as Yes
ReplyDeleteOops:
incidentally, turnout filters normally benefit Yes as Yes on balance seem more motivated..
But I agree about the hunting for a higher yes vote, both James and Scottish Skier tend to do it at times.
ReplyDeleteJust as the BBC's Prof C. seeks a No vote.
Or did he ever criticise that very disturbing / leading ICM question just before the Y/N in their May poll for example?
Of course a real scientist seeks to disprove their theory and this is in a way more important than proving it.
To disprove, we need to find polls which show Scots trust Westminster more than Holyrood, that they believe devo max is coming with a No, that they'd vote in majority to join the union today, that they think the Yes campaign isn't effective and No are, that in identity they are more British than Scottish, that they secretly support the union and have maybe, in the past, e.g. lied to pollsters and said SNP when they intended Labour out of ICM rule Britannia style patriotic shyness in the face of nasty minority nat abuse etc.
I've not had much success here. Well, none really.
So alongside TNS, it would appear that the DKs are starting to break, but at best it's 50/50 - not great news, is it?
ReplyDeleteMy terrible affliction that is confirmation bias shows that the DK lost entirely to Yes in this poll.
38.9(+2.9)% Yes
42.0(nc)% No
19.9(-2.9)% DK
And for balance, survation's turnout weighted headline numbers ex DK:
ReplyDelete47.1 (+0.5)% Yes
52.9 (-0.5)% No
I believe.
On confirmation bias:
ReplyDeleteI don't see confirmation bias, I see analysis that is not just based on this set of polls but also on previous elections history of polls.
Now you could say that is confirmation of my bias, or you could say that you are looking for your bias (of confirmation bias) to be confirmed!
Either way, thanks all, because what this basically does is remind us all - Yes and No - that there is all to play for.
The reason why I like this blog is that all opinions are challenged and nobody takes the huff, keep it up
ReplyDelete