Monday, June 2, 2014

Pro-independence campaign make crucial breakthrough in new Ipsos-Mori telephone poll

Three months ago, Ipsos-Mori published what to date was their only poll for public consumption in referendum year so far.  It baffled many observers by showing a resilience in the already inflated No lead that was extremely difficult to reconcile with the picture painted by virtually all other pollsters.  But a number of people still felt that the dam had to break eventually, even at Ipsos-Mori, and it appears that has finally happened.  The new poll for STV shows a reduction in the No lead since early March of no less than 7.2%.

Should Scotland be an independent country?

Yes 36.3% (+4.2)
No 53.6% (-3.0)

When Don't Knows are excluded, the movement looks even more dramatic, with Yes smashing through the psychological 40% threshold for the first time in any Ipsos-Mori poll conducted during the campaign so far -

Yes 40.4% (+4.2)
No 59.6% (-4.2)

The fieldwork is up to date, and conducted since the European elections.  It's possible, therefore, that this poll is the first concrete sign that revulsion towards UKIP's success south of the border has translated directly into a boost for Yes.  However, we can't assume that's the case, because it's been so long since the last poll, and we could just be catching up belatedly with a swing to Yes that happened weeks ago, or that has been taking place gradually on an ongoing basis.  It's also possible that the March poll was an out-and-out rogue result, and that some of the swing happened even earlier than that.

Make no mistake, though - this is the most important breakthrough that Yes have made at any stage in the campaign so far, with the possible exception of the steady progress they've made in the TNS-BMRB series.  What makes Ipsos-Mori special is that they're the only pollster that conducts referendum polls by telephone - all of the others use volunteer online panels, apart from TNS-BMRB, who use the old-fashioned face-to-face approach.  For as long as Ipsos-Mori remained totally out of line with the online pollsters, there was a genuine possibility that they were getting it right and everyone else was getting it wrong.  Now that they have converged with the norm to some extent (albeit emphatically only to an extent), we can put that worry to one side - and the No campaign will have to live without a comfort blanket that they have become extremely accustomed to.

The fact that the fieldwork for this poll was conducted well after the most recent poll from ICM is a further small indication that the increase in the No lead reported by ICM may well have been an artifact of the margin of error (or of the bizarre change in methodology), rather than something real.  We've now had three different pollsters (Panelbase, Survation and Ipsos-Mori) that have failed to replicate the ICM trend, so we're getting closer to the point where it may be safe to stop worrying about that poll.

Incidentally, Ipsos-Mori have as usual headlined the numbers for those respondents who say they are certain to vote.  Among the entire sample, the reduction in the No lead is in fact 1% greater on the rounded figures -

Yes 34% (+5)
No 52% (-3)

As far as I can see we haven't yet been told what the numbers are on that measure after Don't Knows are stripped out, but a rough calculation suggests it might well be Yes 40% (+5), No 60% (-5).  That would mean that a full third of the No lead has been wiped out since the last poll.  [UPDATE : I've just found the datasets, and it is indeed Yes 40% (+5), No 60% (-5)].

Leaving aside his utterly laughable claim that the Yes side would be "panicking" about a poll showing a 7% reduction in the gap (!), the No camp's embarrassment of a campaign chief Blair McDougall seems to be drawing particular comfort from the fact that the relatively small number of undecided respondents in this poll broke more for No when they were pressed.  However, that misses the point to some extent - because the 42% Yes vote among undecided voters who expressed a leaning is slightly higher than the 40% Yes vote among decided voters, that actually gives a slight boost to Yes if you add the undecideds in.  So absolutely no joy for Blair there.

And what does this tell us about the notorious secret Ipsos-Mori mega-poll commissioned by the UK government?  Not a huge amount, other than the fact that this result is perfectly consistent with the strong rumour that the mega-poll detected significant movement towards Yes.  Without knowing what the exact figures were, though, it's obviously impossible to say whether there was a really big surge towards Yes that has since receded slightly, or whether the surge was roughly identical to what we are looking at now, in which case it has been successfully consolidated.  It's also worth pointing out that we shouldn't take it as read that Ipsos-Mori used an absolutely identical methodology in the mega-poll to the one they use for their published polls, so even if we knew what the figures were they might still not be directly comparable.

The usual disclaimers about Ipsos-Mori apply here - they appear to only interview people by landline (we don't know that for a fact, but any evidence for mobile fieldwork is completely lacking), which means that they may be reaching a small 'c' conservative sample who are disproportionately less likely to be pro-independence.  But of course, there are equivalent caveats about online pollsters, who may be reaching people who are unusually politically aware.  So it really is very difficult to judge where the truth lies, although the face-to-face approach of TNS-BMRB offers a very useful additional dimension.

*  *  *

SCOT GOES POP POLL OF POLLS

After the recovery in the No lead in the last update of the Poll of Polls that was almost entirely caused by the somewhat questionable ICM poll, normal service has now been resumed, with the No lead slipping back to a position closer to the all-time low.

MEAN AVERAGE (excluding Don't Knows) :

Yes 42.8% (+0.8)
No 57.2% (-0.8)

MEAN AVERAGE (not excluding Don't Knows) :

Yes 35.7% (+0.7)
No 47.8% (-0.5)

MEDIAN AVERAGE (excluding Don't Knows) :

Yes 42.3% (n/c)
No 57.7% (n/c)

(The Poll of Polls is based on a rolling average of the most recent poll from each of the pollsters that have been active in the referendum campaign since September 2013, and that adhere to British Polling Council rules. At present, there are six - YouGov, TNS-BMRB, Survation, Panelbase, Ipsos-Mori and ICM. Whenever a new poll is published, it replaces the last poll from the same company in the sample. Changes in the Poll of Polls are generally glacial in nature due to the fact that only a small portion of the sample is updated each time.)

Here are the long-term trend figures, with the updates prior to Easter recalculated to exclude the inactive pollster Angus Reid...

The No campaign's lead in the Poll of Polls mean average (not excluding Don't Knows) :

Sep 2013 - 21.6%
Sep 2013 - 21.4%
Sep 2013 - 19.4%
Oct 2013 - 18.8%
Oct 2013 - 18.4%
Oct 2013 - 18.2%
Nov 2013 - 18.4%
Nov 2013 - 18.0%
Dec 2013 - 17.0%
Dec 2013 - 16.8%
Dec 2013 - 16.4%
Jan 2014 - 14.4%
Jan 2014 - 14.2%
Jan 2014 - 14.2%
Jan 2014 - 15.2%
Feb 2014 - 15.0%
Feb 2014 - 15.5%
Feb 2014 - 15.5%
Feb 2014 - 13.7%
Feb 2014 - 13.3%
Feb 2014 - 14.2%
Mar 2014 - 14.2%
Mar 2014 - 14.5%
Mar 2014 - 14.5%
Mar 2014 - 14.7%
Mar 2014 - 13.8%
Mar 2014 - 13.0%
Mar 2014 - 12.5%
Apr 2014 - 12.5%
Apr 2014 - 12.7%
Apr 2014 - 12.7%
Apr 2014 - 12.3%
Apr 2014 - 11.4%
May 2014 - 11.2%
May 2014 - 11.2%
May 2014 - 11.5%
May 2014 - 13.3%
Jun 2014 - 12.1%

21 comments:

  1. YES have now broke the 40% barrier exDKs with every BPC pollster. This poll is very good news.

    ReplyDelete
  2. James, do you have data on the 1999 referendum in term of female vs male vote? Maybe for the 2007 election too? I understand women lagged behind men for 2011, but eventually voted in nearly the same way.

    If I was BT I'd be bricking it that MORI is showing parity amongst men. Women are being presented with exactly the same information after all.

    ReplyDelete
  3. I heard a rumour that Yes was 36% in the 'Secret Poll' James.

    ReplyDelete
  4. SS : No, I'm afraid I don't have those figures. I don't know how far back the archives go at the Ipsos-Mori and YouGov websites, but it may be possible to trace the 2007 numbers.

    ReplyDelete
  5. Oh those were the days:

    http://www.icmresearch.com/data/media/pdf/1998_june_scotsman_scotland_opinion_poll.pdf

    56% Yes
    35% No

    But no male / female breakdown.

    Do they not have election study info on how people actually voted in elections are referenda?

    ReplyDelete
  6. Probably, but the 1997 referendum is so long ago that I'm not sure the information (if it exists) would be available online.

    ReplyDelete
  7. Have you seen this... links between the Herald, predatory financial services and dodgy referendum polls...The Carrington Dean Scottish Teen Money Survey Scam

    https://pinkindustry.wordpress.com/

    ReplyDelete
  8. Does this poll really only have the Labour yes vote on 11%? Surely not right? Others seem to be more like over 25%?

    They are way under on the national identity thing again too. About 15% too British?

    I bet a re-weighting to 25% labour=yes and NI to the census would have yes and no neck and neck.

    ReplyDelete
  9. Excellent progress. You have to feel sorry for Better Together and their tame media pundits trying to explain this kind of polling movement away.

    No doubt the out of touch PB tories think it's a triumph for No, though the PB herd likely still haven't realised the tories came third in the EU elections. To be fair politicalbetting is now so dominated by witless right wingers and racists shrieking at each other it's fairly obvious why they are so comically out of touch.

    ReplyDelete
  10. Correcting on SSAS 10yr Nat ID gives something like this:

    39% Yes
    46% No

    Which would look more like online polls.

    Most folk in my age / friends / colleagues group (I'm 37 now James - don't remind me that 1997 was ages ago...seems only yesterday ;-) ) don't use a landline any more yet are predominantly Yes.

    We have one because you need to go to the end of the garden and stick your arm in the air to get a mobile signal (rural borders). Only answer it though if it's a number we know as we really, really don't need a conservatory.

    ReplyDelete
  11. Hi Scottish Skier, used to follow your posts on wings but haven't seen any for a while. Could you expand a bit on how you are seeing things just to get another angle on it all? Thanks.

    ReplyDelete
  12. Here in the Central part of Dundee it is interesting to see Yes window posters starting to appear in windows that don't have a history of any SNP posters.

    ReplyDelete
  13. Since we know potty Carlotta and the other PB tory twits are reading this.. the reason there are no Yes supporters commenting on PoliticalBetting is that it's been turned into a far-right zoo by your cowardly hero TSE. We know the tory mod TSE actually BANS left of centre and scottish posters lest they correct herd stupidity and his laughable tory bias too much.

    This would the same tory moderator TSE who lied about his own baby dying to try and welch out of a bet. As his own email here proves.

    http://scotgoespop.blogspot.co.uk/2014/02/political-betting-moderator-screaming.html

    As Carlotta would no doubt say ...bless...

    Never mind PB tories,

    As the incompetent fop Cameron's close friend Rebekah Brooks used to say..

    Lots of Love

    LOL

    Tick, tock ;-)

    ReplyDelete
  14. Although this certainly isn't a bad poll for Yes, I still find it hard to see a genuinely optimistic picture here. If this is indeed an accurate reflection of public opinion then it still translates into a nearly 20% gap between No and Yes among decided voters. That's massive in all honesty when we're a little over three months away from the vote. It's the sort of lead that can only really be turned around by a huge swing toward Yes (a monumental gaffe from the No side, a surge of patriotism during the Commonwealth Games, etc.)

    If on the other hand it isn't accurate then it's difficult to read any meaningful trend into the data. There could be some systematic bias through the use of phone polling and that might imply this trend will be repeated in other polls, but really you can't draw that conclusion until we see other post-EP election polls from other companies.

    Either way there needs to be a large gain in support soon or it's looking like a serious long shot.

    ReplyDelete
  15. if you look at the biggest poll that was last month's euro elections - pro independence parties polled just 37 % while pro union polled 63% out of a total of over 1,300,000 approximately.

    ReplyDelete
  16. Steven : I couldn't disagree more, quite honestly. That's exactly the kind of knee-jerk reaction to the headline numbers of an individual poll that I've tried to guard against by introducing the Poll of Polls. In a campaign where there are sustained, enormous divergences between the results shown by different polling companies, it's just silly beyond belief to effectively say : "OK, the most recent individual poll shows a 60-40 split, so until I see something different I have to assume we're 20 points behind." We don't need to wait for further supporting evidence that Ipsos-Mori are the most No-friendly of the six BPC-pollsters - that's been consistently proven throughout the campaign. To give just one example, in September Ipsos-Mori were showing a 28-point No lead, but ICM were showing only a 17-point No lead, and Panelbase were showing only a 10-point No lead. To point out the bleedin' obvious, an 18-point gap between the most No-friendly and the most Yes-friendly pollsters is not trivial. That doesn't mean that Ipsos-Mori are guilty of "systemic bias", but it does mean that we won't have a clue who is getting it right and who is getting it wrong until the votes are counted - and it's perfectly possible that by polling day some firms will be showing a Yes lead and some will be showing a No lead.

    Nor, incidentally, is it remotely the case that a Yes vote would be a "serious long shot" if Ipsos-Mori happened to be right. A referendum is not an election campaign - big swings of opinion are much more likely because people don't have entrenched voting patterns (indeed, how much more evidence do you need than an 8.4% reduction in the No lead since the last Ipsos-Mori poll?). But even if this WAS a regular election, we wouldn't need to look any further than the last Holyrood campaign to find an example of exactly such a huge swing occurring over the space of a few short weeks. That happened without needing any "monumental gaffes" from Labour. (OK, Iain Gray took shelter in a sandwich shop, but that was scarcely the Profumo affair.)

    ReplyDelete
  17. Anon : Yawn. I presume by that token Scotland has already voted for independence by voting for pro-Yes parties in the 2011 Holyrood list vote?

    And you might want to check what percentage of UKIP voters from last month are planning to vote Yes in September - although prepare yourself for a shock first.

    ReplyDelete
  18. @James That's an oddly irate response to a fairly mundane observation. You've also effectively "debunked" several points that I didn't even come close to arguing.

    I'm fully aware, for instance, that Ipsos-Mori have shown consistently higher shares for No than other polling companies. I'm also fully aware that we have no way of telling at this juncture which polling company is "getting it right" (if any).

    The point I was making is that either Ipsos-Mori are "getting it wrong" (in which case we can't assume a change in their polling has any real impact) or they aren't (in which case No has a sizeable lead a few months out of the referendum). At no point did I argue anything even vaguely approximating "ok, one poll puts No a long way ahead, therefore we have no chance".

    To elaborate on the first point, there are two main types of error that can lead to misleading results. The first is random measurement error, in which the gathering of responses leads to incoherent errors that have no particular pattern. In this case a poll going up or down isn't representative of anything because if you feed gibberish in you'll get gibberish out.

    The second type of error is a systematic error. Given the question being asked here is broadly similar across different polling companies, the main error would likely be sampling/weighting which creates a systematic bias in the results - such as only asking telephone respondents when telephone respondents, on average, are more likely to vote Yes/No than society at large.

    If that's the case then it would be a mistake to assume that a change in this body of respondents will necessarily have a parallel effect across all respondents. It might do, but you simply can't know for sure until the other (more accurate under this assumption) polling confirms it. Ipsos-Mori is the only polling company that has observed this rise in Yes in recent weeks, so until we see that in other polls it's simply too early to draw a conclusion.

    On the broader point, of course opinions can change rapidly - I acknowledged that above - but without being too pessimistic about it we really are at a stage where it's going to take a major development of some kind to swing opinion around to Yes. It happened in Quebec in 1995 with Bouchard's late intervention and I genuinely think that's what Yes needs now.

    ReplyDelete
  19. "That's an oddly irate response to a fairly mundane observation."

    I wasn't "irate", but if you interpreted it in that way it's probably because we differ on what constitutes a "mundane observation". You took this rather good poll for Yes and used it as supporting evidence that Yes are unlikely to win - in my view that's bizarre and fundamentally misconceived, but whatever else it is it's certainly not "mundane" and uncontroversial.

    "The point I was making is that either Ipsos-Mori are "getting it wrong" (in which case we can't assume a change in their polling has any real impact)"

    Nope. That makes no sense whatever. The pollsters may be miles apart in terms of the headline numbers, but with a few important exceptions they've broadly agreed with each other about the trend - so regardless of whether Ipsos-Mori's methodology proves to be the most accurate, it does matter when they show a 4.2% swing to Yes. They're not "feeding gibberish in".

    "Ipsos-Mori is the only polling company that has observed this rise in Yes in recent weeks, so until we see that in other polls it's simply too early to draw a conclusion."

    I don't understand what you mean by that - this is the first Ipsos-Mori poll for three months. Lots of polls from other firms have been published over the last three months, some quite recently.

    "At no point did I argue anything even vaguely approximating "ok, one poll puts No a long way ahead, therefore we have no chance"."

    Really? Just read your final paragraph!

    ReplyDelete
  20. Have to say I find it a little suspicious that the poll figures - after a break of 3 months and one that may have been suppressed - come out at precisely Yes 40.4% and
    No 59.6%

    That is exactly, and only just, enough to keep the headline figure at 60-40, which is the figure NO campaigners often want to believe and insist will be the result.

    A shift of any more than 0.1% would have broken a psychological barrier by being rounded up to 41/59.

    May be entirely coincidence, of course. Just looks suspicious.

    ReplyDelete