So much for Euan McColm's hints that there was bad polling news for Yes coming tonight. Hot on the heels of the sensational ICM poll that put the Yes campaign 8% ahead, we now finally have the first signs that what we used to call the "most Yes-friendly pollster" is picking up the same pro-Yes swing as everyone else (with the possible exception of Survation).
Should Scotland be an independent country?
Yes 49.4% (+1.9)
No 50.6% (-1.9)
For obvious reasons that will be rounded down to 49/51 for publication, but in fact those results are virtually identical to the TNS-BMRB poll, which was rounded up to 50/50.
With the exception of one poll from last September that is generally disregarded because of an unusual question sequence, this is the best poll for Yes that Panelbase have ever published. Although they've tended to produce much more modest shifts than other pollsters, it has to be borne in mind that they've made a number of small methodological changes which have probably had a cumulative No-friendly effect, so the progress that Yes have had to make to reach 49% may well be slightly more substantial than would appear to be the case at a casual glance.
Of course there was a time not too long ago when a poll like this would have been greeted by No supporters with a dismissive "it's only Panelbase" and a shrug of the shoulders. That won't work anymore - Panelbase have well and truly 'slipped back into the pack', and their results now look positively mainstream. Three out of seven BPC pollsters now agree that Yes are on 49% or higher, and five out of seven agree that they are on 47% or higher. (One of the other two doesn't really count, because Ipsos-Mori haven't published a poll since the evening of the first leaders' debate.)
I'm now going to embark on the mammoth task of analysing the other three of today's four polls (or of falling asleep - whichever happens first), but to make matters easier I'll start by copying-and-pasting something I've just written on the previous post about Yes taking the lead with ICM...
I just want to deal with a point about the ICM poll that has already been discussed at length in the comment section below and elsewhere. It is categorically not the case that it should be regarded as a lesser poll because it had a slightly lower sample size than usual (705 as opposed to the more typical 1000). According to the very handy MOE calculator on the ComRes website, the margin of error for a poll of 700 respondents is approximately 3.7% - which is only fractionally higher than the 3% margin of error for a poll of 1000. Someone suggested below that I have said in the past that, by definition, polls of this size cannot be regarded as statistically reliable. I've never said that, because it isn't true. The only thing I can think of is that I put a health warning on YouGov "polls" of roughly this size in the run-up to the European elections, but that's because they weren't proper polls - they were aggregates of subsamples from GB-wide polls. The key factor in determining whether a poll is legitimate is not the sample size (unless it is absurdly small) but rather whether it was properly weighted. As far as I know, this one was weighted in exactly the same way as any other ICM online poll would be - I'm quite sure Professor Curtice would have been the first to flag it up if that had not been the case. So it should be regarded as a legitimate ICM online poll in exactly the same way as any other ICM online poll.
Indeed, we've had smaller polls than this during the campaign - two of the three Angus Reid polls that were published last year had samples of 500 or so, and although that fact was noted in passing, it wasn't perceived as significantly detracting from the credibility of the results. In the US, samples of 500 would be regarded as fairly routine, and it would certainly be seen as very odd to use that as an excuse for ignoring any given poll. And what about the ICM poll with a sample of 500 which the media seized upon as 'absolute proof' that Alistair Darling had defeated Alex Salmond in the first debate? I don't recall Euan McColm casting doubts on the reliability of that one. (I did, as it happens, as did Professor Curtice to some extent, but that was mostly because of the extreme upweighting that had gone on.)
It's therefore extremely troubling that the title Curtice uses for his blogpost on the ICM poll is "ICM Put Yes Ahead - Perhaps". Does the "perhaps" refer to the 3.7% margin of error? If so, shouldn't any verdict on any standard poll of 1000 people also automatically have the word "perhaps" attached to the title, because all of those polls have margins of error of 3%? Or is there some kind of mystical gulf between the numbers 3 and 3.7 that I've failed to comprehend so far? It just seems like a ludicrous double-standard. I note that a report on the BBC website is mentioning all of the other polls today, but not the ICM one - I can't help wondering if that's been caused by a misinterpretation of Curtice's title as meaning that this is somehow not a "proper poll". By the way, I have no complaint about Curtice's decision to add this poll to his Poll of Polls, but to give it a slightly lower weighting in proportion with the sample size - that seems perfectly reasonable to me. But it should also be seen absolutely, unambiguously as an endorsement of the poll's legitimacy, because that's exactly what it is.
You probably don't need me to point out that you would have to go to the extreme end of a 3.7% margin of error to take this poll to a result that does not have Yes clearly in the lead. Given the strength of the evidence from other pollsters that No are either slightly ahead or level, perhaps we do need to assume that ICM have produced a result that is at the extreme end of the margin of error, but so what? We could be forgiven for making exactly the same assumption about today's Survation poll, which is showing a lower Yes vote than others.
On the previous thread, Keaton makes the point that the ICM poll looks very much like an extreme outlier. I wouldn't necessarily disagree with that, but that doesn't mean that its publication shouldn't affect our perception of the state of play. Look at it this way : other recent polls say that the Yes vote is somewhere between 46 and 51. If the true position was closer to 46 than to 51, you'd expect that any extreme outlier poll that came along would put them on something like 43, rather than 54. So if nothing else, ICM have certainly bolstered the impression of a race that is very close to being even-stevens. A crude average of the rounded numbers from today's four polls gives a position of Yes 48.5%, No 51.5%.
It should also give us heart that the ICM fieldwork started one day later than the same firm's telephone poll started, and finished one day later than the telephone poll finished. That may not seem like a huge difference, but the scare stories have literally been ratcheting up with every passing day, so it's important to have some kind of evidence that the electorate resisted them until at least Friday. Unfortunately, the most apocalyptic of all the 'warnings' - very much of the "YOUR CHILDREN WILL ALL DIE IN HORRIBLE AGONY IF YOU VOTE YES" variety - is not really factored into any of the polls so far, so we have to bear that in mind. It's hard to believe that noble, selfless intervention into our democratic process won't have had any negative effect at all, but we'll see.
Let's turn now to the Survation telephone poll, which of course was commissioned by the No campaign, and only published after they decided the numbers were sufficiently to their liking (heaven only knows how many unpublished polls they've commissioned over the last couple of weeks). As Calum Findlay pointed out straight away, Yes only ended up being rounded down to 46% by the tiniest of margins, and it's intriguing to speculate whether the poll would ever have seen the light of day if the coin toss had gone the other way, and Yes had been rounded up to 47%. My guess is that it wouldn't. In any case, it was only on the turnout-weighted figures that Yes were rounded down - among the whole sample, the poll shows a position of Yes 46.7%, No 53.3%.
It's also somewhat chastening for the No campaign to find themselves voluntarily publishing a poll which shows that undecided voters are more likely to break for Yes. Whatever happened to the claim that the recent swing to Yes had only occurred because the undecideds who were willing to "take a leap" had already done so, while the remaining hard-core of undecideds were heavily No-leaning? When 'undecided leaners' are added to the voting intention figures for the whole sample (there isn't enough information to add them to the turnout-weighted figures), the position improves to Yes 47.1%, No 52.9%.
Although the Opinium poll was, to use Kenny Farquharson's favourite term, "independently commissioned" and would have been published whatever the results had been, it shows quite a similar story to Survation, in that rounding has made the published figures less favourable to Yes. The unrounded figures are Yes 47.4%, No 52.6%, and without the very strict Ipsos-Mori-style turnout filter being applied, they would be Yes 47.9%, No 52.1%.
* * *
REQUIRED SWINGS
Swing required for 1 out of 7 pollsters to show Yes ahead or level : 0.0%
Swing required for 3 out of 7 pollsters to show Yes ahead or level : 0.5%
Swing required for 4 out of 7 pollsters to show Yes ahead or level : 2.0%
Swing required for 5 out of 7 pollsters to show Yes ahead or level : 2.5%
Swing required for 6 out of 7 pollsters to show Yes ahead or level : 3.5%
Swing required for 7 out of 7 pollsters to show Yes ahead or level : 7.0%
* * *
SCOT GOES POP POLL OF POLLS
Opinium are a fully-fledged BPC-affiliated polling firm, so with this update of the Poll of Polls we move back to using an average of seven firms for the first time since Angus Reid were removed in the spring due to inactivity.
MEAN AVERAGE (excluding Don't Knows) :
Yes 48.1% (+1.1)
No 51.9% (-1.1)
MEAN AVERAGE (not excluding Don't Knows) :
Yes 43.6% (+2.1)
No 47.1% (+0.3)
MEDIAN AVERAGE (excluding Don't Knows) :
Yes 47.9% (+0.3)
No 52.1% (-0.3)
(The Poll of Polls is based on a rolling average of the most recent poll from each of the pollsters that have been active in the referendum campaign since September 2013, and that adhere to British Polling Council rules. At present, there are seven - YouGov, TNS-BMRB, Survation, Panelbase, Ipsos-Mori, Opinium and ICM. Whenever a new poll is published, it replaces the last poll from the same company in the sample. Changes in the Poll of Polls are generally glacial in nature due to the fact that only a small portion of the sample is updated each time.)
Saturday, September 13, 2014
Yes campaign open up eight-point lead in stunning poll from "gold standard" ICM
Just one day after their extraordinary telephone poll that put Yes within 1% of victory, the UK's "gold standard" polling organisation ICM have gone one better by producing an online poll showing Yes in an outright lead - and a lead of eight points, no less. The percentage changes listed below are measured from the last ICM online poll, because that's the one it's directly comparable to,
Should Scotland be an independent country?
Yes 54% (+9)
No 46% (-9)
There's also a new Opinium poll out tonight - no percentage change figures are possible for this one, because it's the first referendum poll Opinium have produced.
Yes 47.4%
No 52.6%
Since I began this post, a Panelbase poll has also been published, so I'm going to analyse all of today's four polls in a fresh post. Before I do that, though, I just want to deal with a point about the ICM poll that has already been discussed at length in the comment section below and elsewhere. It is categorically not the case that it should be regarded as a lesser poll because it had a slightly lower sample size than usual (705 as opposed to the more typical 1000). According to the very handy MOE calculator on the ComRes website, the margin of error for a poll of 700 respondents is approximately 3.7% - which is only fractionally higher than the 3% margin of error for a poll of 1000. Someone suggested below that I have said in the past that, by definition, polls of this size cannot be regarded as statistically reliable. I've never said that, because it isn't true. The only thing I can think of is that I put a health warning on YouGov "polls" of roughly this size in the run-up to the European elections, but that's because they weren't proper polls - they were aggregates of subsamples from GB-wide polls. The key factor in determining whether a poll is legitimate is not the sample size (unless it is absurdly small) but rather whether it was properly weighted. As far as I know, this one was weighted in exactly the same way as any other ICM online poll would be - I'm quite sure Professor Curtice would have been the first to flag it up if that had not been the case. So it should be regarded as a legitimate ICM online poll in exactly the same way as any other ICM online poll.
Indeed, we've had smaller polls than this during the campaign - two of the three Angus Reid polls that were published last year had samples of 500 or so, and although that fact was noted in passing, it wasn't perceived as significantly detracting from the credibility of the results. In the US, samples of 500 would be regarded as fairly routine, and it would certainly be seen as very odd to use that as an excuse for ignoring any given poll. And what about the ICM poll with a sample of 500 which the media seized upon as 'absolute proof' that Alistair Darling had defeated Alex Salmond in the first debate? I don't recall Euan McColm casting doubts on the reliability of that one. (I did, as it happens, as did Professor Curtice to some extent, but that was mostly because of the extreme upweighting that had gone on.)
It's therefore extremely troubling that the title Curtice uses for his blogpost on the ICM poll is "ICM Put Yes Ahead - Perhaps". Does the "perhaps" refer to the 3.7% margin of error? If so, shouldn't any verdict on any standard poll of 1000 people also automatically have the word "perhaps" attached to the title, because all of those polls have margins of error of 3%? Or is there some kind of mystical gulf between the numbers 3 and 3.7 that I've failed to comprehend so far? It just seems like a ludicrous double-standard. I note that a report on the BBC website is mentioning all of the other polls today, but not the ICM one - I can't help wondering if that's been caused by a misinterpretation of Curtice's title as meaning that this is somehow not a "proper poll". By the way, I have no complaint about Curtice's decision to add this poll to his Poll of Polls, but to give it a slightly lower weighting in proportion with the sample size - that seems perfectly reasonable to me. But it should also be seen absolutely, unambiguously as an endorsement of the poll's legitimacy, because that's exactly what it is.
You probably don't need me to point out that you would have to go to the extreme end of a 3.7% margin of error to take this poll to a result that does not have Yes clearly in the lead. Given the strength of the evidence from other pollsters that No are either slightly ahead or level, perhaps we do need to assume that ICM have produced a result that is at the extreme end of the margin of error, but so what? We could be forgiven for making exactly the same assumption about today's Survation poll, which is showing a lower Yes vote than others.
Further analysis of all of today's four polls, plus a Poll of Polls update, will appear HERE.
Should Scotland be an independent country?
Yes 54% (+9)
No 46% (-9)
There's also a new Opinium poll out tonight - no percentage change figures are possible for this one, because it's the first referendum poll Opinium have produced.
Yes 47.4%
No 52.6%
Since I began this post, a Panelbase poll has also been published, so I'm going to analyse all of today's four polls in a fresh post. Before I do that, though, I just want to deal with a point about the ICM poll that has already been discussed at length in the comment section below and elsewhere. It is categorically not the case that it should be regarded as a lesser poll because it had a slightly lower sample size than usual (705 as opposed to the more typical 1000). According to the very handy MOE calculator on the ComRes website, the margin of error for a poll of 700 respondents is approximately 3.7% - which is only fractionally higher than the 3% margin of error for a poll of 1000. Someone suggested below that I have said in the past that, by definition, polls of this size cannot be regarded as statistically reliable. I've never said that, because it isn't true. The only thing I can think of is that I put a health warning on YouGov "polls" of roughly this size in the run-up to the European elections, but that's because they weren't proper polls - they were aggregates of subsamples from GB-wide polls. The key factor in determining whether a poll is legitimate is not the sample size (unless it is absurdly small) but rather whether it was properly weighted. As far as I know, this one was weighted in exactly the same way as any other ICM online poll would be - I'm quite sure Professor Curtice would have been the first to flag it up if that had not been the case. So it should be regarded as a legitimate ICM online poll in exactly the same way as any other ICM online poll.
Indeed, we've had smaller polls than this during the campaign - two of the three Angus Reid polls that were published last year had samples of 500 or so, and although that fact was noted in passing, it wasn't perceived as significantly detracting from the credibility of the results. In the US, samples of 500 would be regarded as fairly routine, and it would certainly be seen as very odd to use that as an excuse for ignoring any given poll. And what about the ICM poll with a sample of 500 which the media seized upon as 'absolute proof' that Alistair Darling had defeated Alex Salmond in the first debate? I don't recall Euan McColm casting doubts on the reliability of that one. (I did, as it happens, as did Professor Curtice to some extent, but that was mostly because of the extreme upweighting that had gone on.)
It's therefore extremely troubling that the title Curtice uses for his blogpost on the ICM poll is "ICM Put Yes Ahead - Perhaps". Does the "perhaps" refer to the 3.7% margin of error? If so, shouldn't any verdict on any standard poll of 1000 people also automatically have the word "perhaps" attached to the title, because all of those polls have margins of error of 3%? Or is there some kind of mystical gulf between the numbers 3 and 3.7 that I've failed to comprehend so far? It just seems like a ludicrous double-standard. I note that a report on the BBC website is mentioning all of the other polls today, but not the ICM one - I can't help wondering if that's been caused by a misinterpretation of Curtice's title as meaning that this is somehow not a "proper poll". By the way, I have no complaint about Curtice's decision to add this poll to his Poll of Polls, but to give it a slightly lower weighting in proportion with the sample size - that seems perfectly reasonable to me. But it should also be seen absolutely, unambiguously as an endorsement of the poll's legitimacy, because that's exactly what it is.
You probably don't need me to point out that you would have to go to the extreme end of a 3.7% margin of error to take this poll to a result that does not have Yes clearly in the lead. Given the strength of the evidence from other pollsters that No are either slightly ahead or level, perhaps we do need to assume that ICM have produced a result that is at the extreme end of the margin of error, but so what? We could be forgiven for making exactly the same assumption about today's Survation poll, which is showing a lower Yes vote than others.
Further analysis of all of today's four polls, plus a Poll of Polls update, will appear HERE.
Yes secure second-highest ever showing in a telephone poll - and it was commissioned by the No campaign
New Survation telephone poll commissioned by the No campaign -
Should Scotland be an independent country?
Yes 46.5%
No 53.5%
That's by quite some distance the second-highest ever Yes vote in a telephone poll, only beaten by yesterday's ICM poll.
This is the first time "Better Together" have ever released straightforward voting intention figures from one of their private polls, and it's fascinating that they've chosen to do it with a result that even a couple of weeks ago would have been regarded as extremely bad and worrying for them. It's probably an indication that they were concerned that the polling narrative was completely running away from them yesterday.
Fascinating too that they chose Survation - have they detected something in that firm's methodology that will work in their favour? We know that Survation still don't weight by country of birth - the evidence is mixed as to whether that's quite such an issue in phone polls, but even the sample for yesterday's ICM poll over-represented English-born people slightly.
One obvious possibility we shouldn't discount is that BT have used their vast resources to commission several private polls from different companies, and have simply published the worst one for Yes (ie. 'publication bias'). Today's figures are, of course, well within the margin of error of a 49% Yes vote - exactly the same as yesterday's poll.
Further analysis of all of today's four polls, plus a Poll of Polls update, will appear HERE.
Should Scotland be an independent country?
Yes 46.5%
No 53.5%
That's by quite some distance the second-highest ever Yes vote in a telephone poll, only beaten by yesterday's ICM poll.
This is the first time "Better Together" have ever released straightforward voting intention figures from one of their private polls, and it's fascinating that they've chosen to do it with a result that even a couple of weeks ago would have been regarded as extremely bad and worrying for them. It's probably an indication that they were concerned that the polling narrative was completely running away from them yesterday.
Fascinating too that they chose Survation - have they detected something in that firm's methodology that will work in their favour? We know that Survation still don't weight by country of birth - the evidence is mixed as to whether that's quite such an issue in phone polls, but even the sample for yesterday's ICM poll over-represented English-born people slightly.
One obvious possibility we shouldn't discount is that BT have used their vast resources to commission several private polls from different companies, and have simply published the worst one for Yes (ie. 'publication bias'). Today's figures are, of course, well within the margin of error of a 49% Yes vote - exactly the same as yesterday's poll.
Further analysis of all of today's four polls, plus a Poll of Polls update, will appear HERE.
A cornucopia of miscellany
While there's a very brief lull in the mayhem, I thought I'd go through my inbox and try to catch up with a few requests that various people have made over the last couple of days.
I've now been asked about seventeen billion times to give a plug to United Queendom, so I'm finally going to give in to the inevitable! It's billed as "Scottish independence as a gay romantic comedy".
Alan Wood asked me to give a mention to his band's referendum anthem Sunrise Over Scotland, although I see Tris has already beaten me to it! While I'm at it, I may as well link to Make Alba Shine, which you might remember I posted about after seeing it being performed live at the Celtic Connections open stage in January.
There's also a poem that I was meant to publish weeks ago, but it completely slipped my mind - a poll must have come along or something. I'll save that for its own post at some point over the next couple of days.
If anyone else has made any requests, or if I promised to publish something and then forgot about it, feel free to remind me - for obvious reasons I'm a bit snowed under at the moment, but I'll do my best!
I'll leave you with a characteristically not-very-impressive photo I took of Blair Jenkins being interviewed (I presume by the unspeakable Kay Burley) in Edinburgh on Wednesday.
I've now been asked about seventeen billion times to give a plug to United Queendom, so I'm finally going to give in to the inevitable! It's billed as "Scottish independence as a gay romantic comedy".
Alan Wood asked me to give a mention to his band's referendum anthem Sunrise Over Scotland, although I see Tris has already beaten me to it! While I'm at it, I may as well link to Make Alba Shine, which you might remember I posted about after seeing it being performed live at the Celtic Connections open stage in January.
There's also a poem that I was meant to publish weeks ago, but it completely slipped my mind - a poll must have come along or something. I'll save that for its own post at some point over the next couple of days.
If anyone else has made any requests, or if I promised to publish something and then forgot about it, feel free to remind me - for obvious reasons I'm a bit snowed under at the moment, but I'll do my best!
I'll leave you with a characteristically not-very-impressive photo I took of Blair Jenkins being interviewed (I presume by the unspeakable Kay Burley) in Edinburgh on Wednesday.
Friday, September 12, 2014
Yes vote surges to 49% in earth-shaking new TELEPHONE poll from "gold standard" ICM
ICM, the firm regarded by many as the UK's "gold standard" polling organisation, have released their first telephone poll of the campaign (all of their previous polls have been conducted online). This is also the first national telephone poll of the campaign to have been conducted by any firm other than Ipsos-Mori, and is the first to record a Yes vote of higher than 42%. To put it mildly, that previous record has just been smashed to smithereens.
Should Scotland be an independent country?
Yes 49%
No 51%
If Don't Knows are taken into account, the figures are...
Yes 40%
No 42%
I haven't given percentage change figures above, because this poll is not directly comparable with any previous ICM poll. However, if we "just as a bit of fun" make the comparison with last month's online ICM poll, then Yes are up 4% after Don't Knows are excluded, and No are obviously down by the same amount. If Don't Knows are left in, Yes are up 2% and No are down 5%, but those numbers are likely to be particularly misleading, due to the probability of telephone polling detecting a greater number of undecideds.
A bigger problem is that the Yes vote in the last ICM poll seemed strangely low to me, given the direction of travel shown at roughly the same time by Panelbase and YouGov. But today's poll even exceeds ICM's previous all-time high for Yes of 48% that was reported on Easter Sunday - and in retrospect that particular poll was clearly an outlier.
This poll represents what American journalists often refer to as a "statistical tie" - meaning there is a greater than 5% chance that the side that appears to be behind is actually in the lead. In this case, the probability of that is quite a bit higher than 5%, but that's only if you look at this poll in perfect isolation. If you take the three polls published over the last 48 hours together, then the evidence is somewhat more compelling that No are still hanging on to a very small lead - but that crucially depends on the assumption that the pollsters are getting their methodology absolutely right. I gather that ICM's Martin Boon will be going on BBC radio this afternoon to once again express his worries that there is no guarantee that they are. Of course that isn't necessarily a good thing for Yes - there's just as much chance that a misconceived methodology might be underestimating the No vote. Indeed, given that he lives and works within a London media environment that regards the idea of Scottish independence as self-evidently absurd, I wouldn't be surprised if Boon places more emphasis on the possibility of the No vote being too low in his polls. But all the same, it's a straightforward fact that if polls are showing a small No lead, any greater uncertainty than usual over whether the methodology is right somewhat increases the statistical chances that Yes may in fact be in the lead.
So if we imagine for the sake of argument that ICM are underestimating Yes, how might that be happening? The ongoing problem of the failure to weight by country of birth isn't so much of an issue in this poll, because although there are slightly too many English-born respondents in the sample, the degree of error is nowhere near as great as it has been in ICM online polls. And in any case, the Yes campaign are doing fantastically well among English-born people in this poll - they stand at an amazing 37% with Don't Knows excluded!
Instead we might look to the issue of Shy Yes Syndrome - which, if it exists, is more likely to rear its ugly head in a telephone poll than in a more anonymous online survey. We've long speculated on this blog that Yes supporters who receive a phone call from an "authority figure" at Ipsos-Mori might be more likely to feel that they have to give the answer that is expected of them - ie. that they're voting No, or at the very least that they don't know how they'll vote. If anything, though, the problem might be even worse with ICM. As it happens, Ipsos-Mori base their call centre operations in Edinburgh, so it's reasonable to assume that many (although not all) of their interviewers will have a Scottish accent. To the best of my knowledge, that isn't the case with ICM, and I think many of us know from our own experience how much harder it is to admit to supporting independence to someone from the south of England, because of the greater chance of encountering hostility or ridicule. I'm quite sure that ICM's interviewers are extremely professional and skilled at putting people at their ease, but even if there's only a very small number of "shy Yesses", that could be more than enough to swing the balance in this poll. But, of course, this is just pure speculation based on no hard evidence.
One thing that I can't quite fathom about this poll is whether mobile numbers were called. We were told yesterday that they had been, but there's no clear mention of that in ICM's explanation of their method. Paradoxically, the Yes campaign might prefer it if only landlines were called, because that might be another reason for seriously wondering if the Yes vote is being underestimated - it's long been thought plausible that lower-income Yes voters are harder to reach via Ipsos-Mori's landline-only approach.
If this poll had been published a week ago today, it would have boasted the highest Yes vote of the campaign so far in any poll from any firm (with the exception of one Panelbase poll that is often disregarded because of an unusual question sequence). But it wouldn't actually have been the highest by all that much - we've been used to seeing Yes at 47% or 48% in a number of online polls from ICM, Survation and Panelbase. So why has it only been in the last few days that the idea of a Yes victory has been taken seriously by the London media? Partly, it's because YouGov (rightly or wrongly) command enormous respect, and until last week they remained stubbornly out of line with the other online firms. But more fundamentally, it's because the only two pollsters who routinely went out into the 'real world' to find a fresh sample for each new poll (TNS-BMRB and Ipsos-Mori) were firmly on the No-friendly end of the spectrum. That led to a lazy assumption that the No campaign's 'real world' lead must be on the higher end of the scale, and that it was only being underestimated because volunteer online polling panels had too many politically committed people on their books.
That comfort blanket for the Abominable No-Men has now well and truly gone out of the window. As Calum Findlay notes in the comments section below, the two most Yes-friendly pollsters at present are TNS-BMRB and ICM - and they both used 'real world' methodology in their most recent poll.
Final thought : one thing that's really quite impressive about the ICM poll is that respondents were asked how they voted in BOTH the 2010 and 2011 elections, and the answers given are uncannily close to the actual results. Labour voters have had to be downweighted, but not by very much, while the SNP were pretty much bang on. So, just for once, there's no danger that faulty recall will have distorted the headline numbers to any great extent.
* * *
REQUIRED SWINGS
Swing required for 1 out of 6 pollsters to show Yes ahead or level : 0.5%
Swing required for 2 out of 6 pollsters to show Yes ahead or level : 1.0%
Swing required for 3 out of 6 pollsters to show Yes ahead or level : 2.0%
Swing required for 4 out of 6 pollsters to show Yes ahead or level : 2.5%
Swing required for 5 out of 6 pollsters to show Yes ahead or level : 3.0%
Swing required for 6 out of 6 pollsters to show Yes ahead or level : 7.0%
* * *
SCOT GOES POP POLL OF POLLS
If you were around yesterday, you'll know that I was pondering how I should take account of the new ICM telephone poll in the Poll of Polls, and that I wanted to make a firm decision before we knew what the results were. I'll be honest - I reached a firm decision, but now I've changed my mind (so sue me!) The reason is that I think this blog's Poll of Polls methodology will lose credibility if I unnecessarily keep giving some weight to an ancient ICM online poll that was conducted well before the second leaders' debate. It's bad enough that I'll be leaving in an even older Ipsos-Mori poll that was conducted before the first debate! So, as normal, I'm just going to completely drop the last ICM poll from the sample and replace it with the new one. If ICM produce an online poll at the weekend that turns out to have been conducted at roughly the same time as the telephone poll, then I'll revert to yesterday's plan to use an average of the two polls for ICM's one-sixth share of the sample.
MEAN AVERAGE (excluding Don't Knows) :
Yes 47.0% (+0.7)
No 53.0% (-0.7)
MEAN AVERAGE (not excluding Don't Knows) :
Yes 41.5% (+0.3)
No 46.8% (-0.9)
MEDIAN AVERAGE (excluding Don't Knows) :
Yes 47.6% (+0.6)
No 52.4% (-0.6)
(The Poll of Polls is based on a rolling average of the most recent poll from each of the pollsters that have been active in the referendum campaign since September 2013, and that adhere to British Polling Council rules. At present, there are six - YouGov, TNS-BMRB, Survation, Panelbase, Ipsos-Mori and ICM. Whenever a new poll is published, it replaces the last poll from the same company in the sample. Changes in the Poll of Polls are generally glacial in nature due to the fact that only a small portion of the sample is updated each time.)
Should Scotland be an independent country?
Yes 49%
No 51%
If Don't Knows are taken into account, the figures are...
Yes 40%
No 42%
I haven't given percentage change figures above, because this poll is not directly comparable with any previous ICM poll. However, if we "just as a bit of fun" make the comparison with last month's online ICM poll, then Yes are up 4% after Don't Knows are excluded, and No are obviously down by the same amount. If Don't Knows are left in, Yes are up 2% and No are down 5%, but those numbers are likely to be particularly misleading, due to the probability of telephone polling detecting a greater number of undecideds.
A bigger problem is that the Yes vote in the last ICM poll seemed strangely low to me, given the direction of travel shown at roughly the same time by Panelbase and YouGov. But today's poll even exceeds ICM's previous all-time high for Yes of 48% that was reported on Easter Sunday - and in retrospect that particular poll was clearly an outlier.
This poll represents what American journalists often refer to as a "statistical tie" - meaning there is a greater than 5% chance that the side that appears to be behind is actually in the lead. In this case, the probability of that is quite a bit higher than 5%, but that's only if you look at this poll in perfect isolation. If you take the three polls published over the last 48 hours together, then the evidence is somewhat more compelling that No are still hanging on to a very small lead - but that crucially depends on the assumption that the pollsters are getting their methodology absolutely right. I gather that ICM's Martin Boon will be going on BBC radio this afternoon to once again express his worries that there is no guarantee that they are. Of course that isn't necessarily a good thing for Yes - there's just as much chance that a misconceived methodology might be underestimating the No vote. Indeed, given that he lives and works within a London media environment that regards the idea of Scottish independence as self-evidently absurd, I wouldn't be surprised if Boon places more emphasis on the possibility of the No vote being too low in his polls. But all the same, it's a straightforward fact that if polls are showing a small No lead, any greater uncertainty than usual over whether the methodology is right somewhat increases the statistical chances that Yes may in fact be in the lead.
So if we imagine for the sake of argument that ICM are underestimating Yes, how might that be happening? The ongoing problem of the failure to weight by country of birth isn't so much of an issue in this poll, because although there are slightly too many English-born respondents in the sample, the degree of error is nowhere near as great as it has been in ICM online polls. And in any case, the Yes campaign are doing fantastically well among English-born people in this poll - they stand at an amazing 37% with Don't Knows excluded!
Instead we might look to the issue of Shy Yes Syndrome - which, if it exists, is more likely to rear its ugly head in a telephone poll than in a more anonymous online survey. We've long speculated on this blog that Yes supporters who receive a phone call from an "authority figure" at Ipsos-Mori might be more likely to feel that they have to give the answer that is expected of them - ie. that they're voting No, or at the very least that they don't know how they'll vote. If anything, though, the problem might be even worse with ICM. As it happens, Ipsos-Mori base their call centre operations in Edinburgh, so it's reasonable to assume that many (although not all) of their interviewers will have a Scottish accent. To the best of my knowledge, that isn't the case with ICM, and I think many of us know from our own experience how much harder it is to admit to supporting independence to someone from the south of England, because of the greater chance of encountering hostility or ridicule. I'm quite sure that ICM's interviewers are extremely professional and skilled at putting people at their ease, but even if there's only a very small number of "shy Yesses", that could be more than enough to swing the balance in this poll. But, of course, this is just pure speculation based on no hard evidence.
One thing that I can't quite fathom about this poll is whether mobile numbers were called. We were told yesterday that they had been, but there's no clear mention of that in ICM's explanation of their method. Paradoxically, the Yes campaign might prefer it if only landlines were called, because that might be another reason for seriously wondering if the Yes vote is being underestimated - it's long been thought plausible that lower-income Yes voters are harder to reach via Ipsos-Mori's landline-only approach.
If this poll had been published a week ago today, it would have boasted the highest Yes vote of the campaign so far in any poll from any firm (with the exception of one Panelbase poll that is often disregarded because of an unusual question sequence). But it wouldn't actually have been the highest by all that much - we've been used to seeing Yes at 47% or 48% in a number of online polls from ICM, Survation and Panelbase. So why has it only been in the last few days that the idea of a Yes victory has been taken seriously by the London media? Partly, it's because YouGov (rightly or wrongly) command enormous respect, and until last week they remained stubbornly out of line with the other online firms. But more fundamentally, it's because the only two pollsters who routinely went out into the 'real world' to find a fresh sample for each new poll (TNS-BMRB and Ipsos-Mori) were firmly on the No-friendly end of the spectrum. That led to a lazy assumption that the No campaign's 'real world' lead must be on the higher end of the scale, and that it was only being underestimated because volunteer online polling panels had too many politically committed people on their books.
That comfort blanket for the Abominable No-Men has now well and truly gone out of the window. As Calum Findlay notes in the comments section below, the two most Yes-friendly pollsters at present are TNS-BMRB and ICM - and they both used 'real world' methodology in their most recent poll.
Final thought : one thing that's really quite impressive about the ICM poll is that respondents were asked how they voted in BOTH the 2010 and 2011 elections, and the answers given are uncannily close to the actual results. Labour voters have had to be downweighted, but not by very much, while the SNP were pretty much bang on. So, just for once, there's no danger that faulty recall will have distorted the headline numbers to any great extent.
* * *
REQUIRED SWINGS
Swing required for 1 out of 6 pollsters to show Yes ahead or level : 0.5%
Swing required for 2 out of 6 pollsters to show Yes ahead or level : 1.0%
Swing required for 3 out of 6 pollsters to show Yes ahead or level : 2.0%
Swing required for 4 out of 6 pollsters to show Yes ahead or level : 2.5%
Swing required for 5 out of 6 pollsters to show Yes ahead or level : 3.0%
Swing required for 6 out of 6 pollsters to show Yes ahead or level : 7.0%
* * *
SCOT GOES POP POLL OF POLLS
If you were around yesterday, you'll know that I was pondering how I should take account of the new ICM telephone poll in the Poll of Polls, and that I wanted to make a firm decision before we knew what the results were. I'll be honest - I reached a firm decision, but now I've changed my mind (so sue me!) The reason is that I think this blog's Poll of Polls methodology will lose credibility if I unnecessarily keep giving some weight to an ancient ICM online poll that was conducted well before the second leaders' debate. It's bad enough that I'll be leaving in an even older Ipsos-Mori poll that was conducted before the first debate! So, as normal, I'm just going to completely drop the last ICM poll from the sample and replace it with the new one. If ICM produce an online poll at the weekend that turns out to have been conducted at roughly the same time as the telephone poll, then I'll revert to yesterday's plan to use an average of the two polls for ICM's one-sixth share of the sample.
MEAN AVERAGE (excluding Don't Knows) :
Yes 47.0% (+0.7)
No 53.0% (-0.7)
MEAN AVERAGE (not excluding Don't Knows) :
Yes 41.5% (+0.3)
No 46.8% (-0.9)
MEDIAN AVERAGE (excluding Don't Knows) :
Yes 47.6% (+0.6)
No 52.4% (-0.6)
(The Poll of Polls is based on a rolling average of the most recent poll from each of the pollsters that have been active in the referendum campaign since September 2013, and that adhere to British Polling Council rules. At present, there are six - YouGov, TNS-BMRB, Survation, Panelbase, Ipsos-Mori and ICM. Whenever a new poll is published, it replaces the last poll from the same company in the sample. Changes in the Poll of Polls are generally glacial in nature due to the fact that only a small portion of the sample is updated each time.)
Thursday, September 11, 2014
Yes campaign are just 2% from victory in remarkable YouGov poll
It was probably always inevitable that there would be some reversion to the mean from YouGov after the firm showed an almost unbelievable 12% swing over the space of a month, but the Yes campaign will take enormous heart from this, their second-highest ever showing from a once firmly No-friendly pollster - higher even than in the YouGov poll just over a week ago that set this campaign alight.
* * *
REQUIRED SWINGS
Swing required for 1 out of 6 pollsters to show Yes ahead or level : 0.5%
Swing required for 2 out of 6 pollsters to show Yes ahead or level : 2.0%
Swing required for 3 out of 6 pollsters to show Yes ahead or level : 2.5%
Swing required for 4 out of 6 pollsters to show Yes ahead or level : 3.0%
Swing required for 5 out of 6 pollsters to show Yes ahead or level : 4.5%
Swing required for 6 out of 6 pollsters to show Yes ahead or level : 7.0%
* * *
SCOT GOES POP POLL OF POLLS
MEAN AVERAGE (excluding Don't Knows) :
Yes 46.3% (-0.7)
No 53.7% (+0.7)
MEAN AVERAGE (not excluding Don't Knows) :
Yes 41.2% (-0.3)
No 47.7% (+0.9)
MEDIAN AVERAGE (excluding Don't Knows) :
Yes 47.0% (-0.2)
No 53.0% (+0.2)
(The Poll of Polls is based on a rolling average of the most recent poll from each of the pollsters that have been active in the referendum campaign since September 2013, and that adhere to British Polling Council rules. At present, there are six - YouGov, TNS-BMRB, Survation, Panelbase, Ipsos-Mori and ICM. Whenever a new poll is published, it replaces the last poll from the same company in the sample. Changes in the Poll of Polls are generally glacial in nature due to the fact that only a small portion of the sample is updated each time.)
* * *
When I heard that George Galloway was taking part in the TV debate at the Hydro, I naturally assumed that the BBC had gone over Better Together's heads to invite him. When it was announced at the start of the programme that Better Together had actually nominated him and he was specifically there on their behalf, my jaw dropped to the floor. OK, it must have been some kind of tactical gambit to appeal to traditional Labour voters, but what kind of message do they think it sends out to put forward the leader of the Scottish Tories, and a far-left politician who was expelled from Labour years ago? Oh well, at least they've got the anti-German vote sewn up now.
Should Scotland be an independent country?
Yes 48% (-3)
No 52% (+3)
From the moment that Survation published their poll yesterday showing only a very tiny swing to Yes, it always seemed likely that there hadn't been any kind of bandwagon effect for the Yes campaign since the weekend - hardly surprising given the mind-boggling onslaught they've had to endure over the last few days from the entire London establishment, and especially the London media. But the big remaining question was - had Yes gone backwards, or had they retained their big gains? There is now strong evidence pointing to the latter, and it is coming from two different pollsters. Survation showed Yes holding firm at an all-time high of 47%, while the variation between the last three YouGov polls (of between 47% and 51% for Yes) is also entirely consistent with a relatively steady position, masked by the absolutely normal 'noise' that is caused by the 3% margin of error. It therefore seems that the voters who have recently been won over to the Yes side are sticking with the Yes side, in spite of the onslaught. The best evidence of that is YouGov's fieldwork dates - this poll didn't even finish until earlier today, and started only on Tuesday.
I've no idea what the private polling and collated canvassing results on both sides are showing, but certainly from the perspective of anyone who just follows this campaign from the published polls, there would have been good reason to wonder if the No campaign were going to achieve a big swing in this poll. Throughout this year, Survation have consistently reported a much higher Yes vote than YouGov, and in some ways it scarcely seemed credible that Survation could show Yes holding steady at 47% without that being followed by Yes slipping below 47% in the subsequent YouGov poll. OK, we've seen convergence, but surely even after convergence you'd expect to see the normal pecking order of Yes-friendly and No-friendly firms remain in place to a more limited degree? It appears not. Two rogue polls in a row from YouGov was just about imaginable, but a third is not - so we can now say it's official. YouGov are no longer a No-friendly pollster, and whether that's in spite of or because of the "Kellner Correction" is something we can only speculate on for now. Indeed, both TNS and YouGov are as of this moment more Yes-friendly than Survation, so it seems that all the old certainties have gone as we contemplate the crucial final few days.
Obviously Yes campaigners will react to the closeness of the polls by stepping up (even further) their efforts to win over undecideds and soft No voters - and that is absolutely the right thing to do, because we don't know exactly how many more people we need. But as of tonight (and I may well revise this opinion when we get the ICM poll tomorrow) we have at last reached the position where it is just about possible that even if the pollsters have their methodology absolutely right, Yes could be in the lead already. Across all firms, the last five polls have reported Yes votes of 48%, 47%, 51%, 48% and 50%. Bearing in mind the margin of error, that means there is a genuine chance that Yes are ahead, although the balance of odds remain substantially in favour of No retaining a very slender advantage.
But things become more interesting still when you take into account the concerns that pollsters themselves (such as Martin Boon) have openly expressed about the limited confidence they can have in the accuracy of their methodology, due to the unusual nature of the contest. If we therefore assume that the real world margin of error is significantly higher than 3%, then the chances that Yes are already in the lead immediately look much more substantial.
In the Poll of Polls below, you'll see that the Yes vote has slipped back slightly - that's simply because I only use one poll per firm, and therefore this new YouGov poll has replaced the one that had Yes in a slight lead. But John Curtice's Poll of Polls has Yes remaining at the heady heights of 48%, and that's partly because his methodology takes into account all of the last three YouGov polls. In the past, I would have said that approach is a weakness, because in a campaign which has been distinguished by different firms diverging from each other markedly, you really have to ensure that the weight given to each firm remains constant over time. However, now that some of the pollsters have converged, the one big strength of the Curtice approach may be making itself felt - ie. that it uses only the most recent polls, and the results therefore aren't artificially 'held back' by the inclusion of polls from several weeks ago. If it can be assumed that there is no remaining bias for Yes or No in the methodology of any pollster, but that there was until very recently, then for the first time the Curtice Poll of Polls may be the best guide to the state of play and to the trends. It remains to be seen if that's the case, though, because it's still an open question whether ICM and Ipsos-Mori (and indeed Opinium, who are apparently about to publish their first referendum poll) will join the general pattern of convergence.
I haven't been closely following the mainstream media reporting of the Survation poll, so I was shocked (although I shouldn't have been) to see a comment suggesting that many journalists made a bogus apples-and-oranges comparison between that poll and the weekend YouGov poll, in order to wrongly claim that the Yes vote had slipped. If that's true, we really must demand consistency - tonight's poll should therefore be compared to yesterday's Survation poll, and not to the previous YouGov poll. That means that Yes should be reported as being 1% up, and No should be reported as being 1% down.
They really can't have it both ways, no matter how much they'd like to.
From the moment that Survation published their poll yesterday showing only a very tiny swing to Yes, it always seemed likely that there hadn't been any kind of bandwagon effect for the Yes campaign since the weekend - hardly surprising given the mind-boggling onslaught they've had to endure over the last few days from the entire London establishment, and especially the London media. But the big remaining question was - had Yes gone backwards, or had they retained their big gains? There is now strong evidence pointing to the latter, and it is coming from two different pollsters. Survation showed Yes holding firm at an all-time high of 47%, while the variation between the last three YouGov polls (of between 47% and 51% for Yes) is also entirely consistent with a relatively steady position, masked by the absolutely normal 'noise' that is caused by the 3% margin of error. It therefore seems that the voters who have recently been won over to the Yes side are sticking with the Yes side, in spite of the onslaught. The best evidence of that is YouGov's fieldwork dates - this poll didn't even finish until earlier today, and started only on Tuesday.
I've no idea what the private polling and collated canvassing results on both sides are showing, but certainly from the perspective of anyone who just follows this campaign from the published polls, there would have been good reason to wonder if the No campaign were going to achieve a big swing in this poll. Throughout this year, Survation have consistently reported a much higher Yes vote than YouGov, and in some ways it scarcely seemed credible that Survation could show Yes holding steady at 47% without that being followed by Yes slipping below 47% in the subsequent YouGov poll. OK, we've seen convergence, but surely even after convergence you'd expect to see the normal pecking order of Yes-friendly and No-friendly firms remain in place to a more limited degree? It appears not. Two rogue polls in a row from YouGov was just about imaginable, but a third is not - so we can now say it's official. YouGov are no longer a No-friendly pollster, and whether that's in spite of or because of the "Kellner Correction" is something we can only speculate on for now. Indeed, both TNS and YouGov are as of this moment more Yes-friendly than Survation, so it seems that all the old certainties have gone as we contemplate the crucial final few days.
Obviously Yes campaigners will react to the closeness of the polls by stepping up (even further) their efforts to win over undecideds and soft No voters - and that is absolutely the right thing to do, because we don't know exactly how many more people we need. But as of tonight (and I may well revise this opinion when we get the ICM poll tomorrow) we have at last reached the position where it is just about possible that even if the pollsters have their methodology absolutely right, Yes could be in the lead already. Across all firms, the last five polls have reported Yes votes of 48%, 47%, 51%, 48% and 50%. Bearing in mind the margin of error, that means there is a genuine chance that Yes are ahead, although the balance of odds remain substantially in favour of No retaining a very slender advantage.
But things become more interesting still when you take into account the concerns that pollsters themselves (such as Martin Boon) have openly expressed about the limited confidence they can have in the accuracy of their methodology, due to the unusual nature of the contest. If we therefore assume that the real world margin of error is significantly higher than 3%, then the chances that Yes are already in the lead immediately look much more substantial.
In the Poll of Polls below, you'll see that the Yes vote has slipped back slightly - that's simply because I only use one poll per firm, and therefore this new YouGov poll has replaced the one that had Yes in a slight lead. But John Curtice's Poll of Polls has Yes remaining at the heady heights of 48%, and that's partly because his methodology takes into account all of the last three YouGov polls. In the past, I would have said that approach is a weakness, because in a campaign which has been distinguished by different firms diverging from each other markedly, you really have to ensure that the weight given to each firm remains constant over time. However, now that some of the pollsters have converged, the one big strength of the Curtice approach may be making itself felt - ie. that it uses only the most recent polls, and the results therefore aren't artificially 'held back' by the inclusion of polls from several weeks ago. If it can be assumed that there is no remaining bias for Yes or No in the methodology of any pollster, but that there was until very recently, then for the first time the Curtice Poll of Polls may be the best guide to the state of play and to the trends. It remains to be seen if that's the case, though, because it's still an open question whether ICM and Ipsos-Mori (and indeed Opinium, who are apparently about to publish their first referendum poll) will join the general pattern of convergence.
I haven't been closely following the mainstream media reporting of the Survation poll, so I was shocked (although I shouldn't have been) to see a comment suggesting that many journalists made a bogus apples-and-oranges comparison between that poll and the weekend YouGov poll, in order to wrongly claim that the Yes vote had slipped. If that's true, we really must demand consistency - tonight's poll should therefore be compared to yesterday's Survation poll, and not to the previous YouGov poll. That means that Yes should be reported as being 1% up, and No should be reported as being 1% down.
They really can't have it both ways, no matter how much they'd like to.
* * *
REQUIRED SWINGS
Swing required for 1 out of 6 pollsters to show Yes ahead or level : 0.5%
Swing required for 2 out of 6 pollsters to show Yes ahead or level : 2.0%
Swing required for 3 out of 6 pollsters to show Yes ahead or level : 2.5%
Swing required for 4 out of 6 pollsters to show Yes ahead or level : 3.0%
Swing required for 5 out of 6 pollsters to show Yes ahead or level : 4.5%
Swing required for 6 out of 6 pollsters to show Yes ahead or level : 7.0%
* * *
SCOT GOES POP POLL OF POLLS
MEAN AVERAGE (excluding Don't Knows) :
Yes 46.3% (-0.7)
No 53.7% (+0.7)
MEAN AVERAGE (not excluding Don't Knows) :
Yes 41.2% (-0.3)
No 47.7% (+0.9)
MEDIAN AVERAGE (excluding Don't Knows) :
Yes 47.0% (-0.2)
No 53.0% (+0.2)
(The Poll of Polls is based on a rolling average of the most recent poll from each of the pollsters that have been active in the referendum campaign since September 2013, and that adhere to British Polling Council rules. At present, there are six - YouGov, TNS-BMRB, Survation, Panelbase, Ipsos-Mori and ICM. Whenever a new poll is published, it replaces the last poll from the same company in the sample. Changes in the Poll of Polls are generally glacial in nature due to the fact that only a small portion of the sample is updated each time.)
* * *
When I heard that George Galloway was taking part in the TV debate at the Hydro, I naturally assumed that the BBC had gone over Better Together's heads to invite him. When it was announced at the start of the programme that Better Together had actually nominated him and he was specifically there on their behalf, my jaw dropped to the floor. OK, it must have been some kind of tactical gambit to appeal to traditional Labour voters, but what kind of message do they think it sends out to put forward the leader of the Scottish Tories, and a far-left politician who was expelled from Labour years ago? Oh well, at least they've got the anti-German vote sewn up now.
Significant swing to Yes in new ComRes poll of the Borders and Dumfries & Galloway
A new ComRes regional poll of one the most No-friendly parts of the country, the Borders and Dumfries & Galloway, has just been released -
Should Scotland be an independent country?
Yes 33% (+3)
No 67% (-3)
Percentage changes in brackets are since the last comparable poll in June.
What I always find interesting about these polls, commissioned by ITV Border, is that they are (very expensively) conducted by telephone, which is incredibly rare in this campaign. Until now, Ipsos-Mori have been the only pollster that has conducted nationwide telephone polls, although ICM are belatedly about to join the fray. It's incredibly frustrating, because ITV Border could have commissioned a nationwide telephone poll for exactly the same price, and actually told us something useful. Instead, we're left scratching our heads, and trying to work out if there's some way of extrapolating from these numbers to get a very rough sense of what they might imply for the national picture.
The only approach I can think of is to look at the 1997 devolution referendum results, in which the Yes vote on the first question was 12.7% lower in the Borders and Dumfries & Galloway than it was across the country, and the Yes vote on the second (tax) question was 14% lower than the national figure. There's obviously no way of knowing if the same differential is holding true in this contest, but if by any chance it is, tonight's poll would imply a national Yes vote of about 46% or 47%, which is a significantly higher figure than recorded in any nationwide telephone poll (all from Ipsos-Mori) to date.
But as Dan Snow's dad used to say, this is "just a bit of fun".
Should Scotland be an independent country?
Yes 33% (+3)
No 67% (-3)
Percentage changes in brackets are since the last comparable poll in June.
What I always find interesting about these polls, commissioned by ITV Border, is that they are (very expensively) conducted by telephone, which is incredibly rare in this campaign. Until now, Ipsos-Mori have been the only pollster that has conducted nationwide telephone polls, although ICM are belatedly about to join the fray. It's incredibly frustrating, because ITV Border could have commissioned a nationwide telephone poll for exactly the same price, and actually told us something useful. Instead, we're left scratching our heads, and trying to work out if there's some way of extrapolating from these numbers to get a very rough sense of what they might imply for the national picture.
The only approach I can think of is to look at the 1997 devolution referendum results, in which the Yes vote on the first question was 12.7% lower in the Borders and Dumfries & Galloway than it was across the country, and the Yes vote on the second (tax) question was 14% lower than the national figure. There's obviously no way of knowing if the same differential is holding true in this contest, but if by any chance it is, tonight's poll would imply a national Yes vote of about 46% or 47%, which is a significantly higher figure than recorded in any nationwide telephone poll (all from Ipsos-Mori) to date.
But as Dan Snow's dad used to say, this is "just a bit of fun".
Poll schedule for tonight
The ones we know about so far are...
6pm : ComRes regional telephone poll of the Borders and Dumfries & Galloway. This is another of those mystifyingly pointless ITV Border polls that will be billed as a "South of Scotland" poll, even though it isn't - well over half the population of the South of Scotland electoral region is excluded.
10pm : YouGov nationwide online poll. On past form the embargo is likely to be broken by someone well before 10pm, though.
No consensus emerged on how I should use the forthcoming ICM telephone poll in the Poll of Polls, so I'm going to take up the suggestion that I should keep ICM's share of the sample at one-sixth, but use an average of their most recent telephone poll and their most recent online poll. A bit complicated, but all of the other possibilities seemed even worse to me.
6pm : ComRes regional telephone poll of the Borders and Dumfries & Galloway. This is another of those mystifyingly pointless ITV Border polls that will be billed as a "South of Scotland" poll, even though it isn't - well over half the population of the South of Scotland electoral region is excluded.
10pm : YouGov nationwide online poll. On past form the embargo is likely to be broken by someone well before 10pm, though.
No consensus emerged on how I should use the forthcoming ICM telephone poll in the Poll of Polls, so I'm going to take up the suggestion that I should keep ICM's share of the sample at one-sixth, but use an average of their most recent telephone poll and their most recent online poll. A bit complicated, but all of the other possibilities seemed even worse to me.
A polite message to "effing" David Cameron
Mr Prime Minister, sir.
The reason I am voting Yes is not that I want to, in your words, "give the effing Tories a kicking".
As you may be aware, this is not a by-election, or a meaningless protest vote based on emotion or anger. The long-term future of this country is at stake next Thursday.
It is for that reason that I am very calmly voting Yes, to ensure that Scotland is never again ruled by a Tory government it didn't vote for.
Can you understand the difference? Take your time.
The reason I am voting Yes is not that I want to, in your words, "give the effing Tories a kicking".
As you may be aware, this is not a by-election, or a meaningless protest vote based on emotion or anger. The long-term future of this country is at stake next Thursday.
It is for that reason that I am very calmly voting Yes, to ensure that Scotland is never again ruled by a Tory government it didn't vote for.
Can you understand the difference? Take your time.
ICM telephone poll poser
Thanks to Xabi on the previous thread for letting me know that ICM are conducting a referendum poll for the Guardian - and unlike the firm's series for Scotland on Sunday, this one is being done by telephone rather than online. Believe it or not, this will be the first national telephone poll of the entire campaign to be conducted by a company other than Ipsos-Mori, who of course have tended to produce much more No-friendly results than the online pollsters, including ICM themselves. It's unclear how much of this divergence is due to the telephone factor itself (and in particular the apparent landline-only nature of the polls), and how much is due to other distinctive features of Ipsos-Mori's methodology, such as failure to weight by past vote recall.
This development poses a problem for this blog's Poll of Polls, which uses the most recent poll from each firm, meaning that an ICM telephone poll will be replacing an ICM online poll in the sample. That could well distort the trend, because there's a good chance that telephone polling will produce different results (my guess is that they'll be more No-friendly, but there's no way of knowing that for sure). The alternative would be to treat "ICM (online)" and "ICM (telephone)" as two separate firms for the purposes of the Poll of Polls - but that might give too much weight to a single firm, particularly given that they'll presumably be using exactly the same methodology other than data collection.
Any thoughts? For the sake of transparency, it would probably be best to make a firm decision well before we actually know what ICM are showing.
Survation appear to be showing Yes on 47.1% - a slight increase on the last poll
Apologies for taking so long to get some analysis of the Survation poll up. I was out and about today (in fact I passed Blair Jenkins as he was doing a TV interview on the Mound!) and I was checking my mobile phone so much to find out what was going on with the poll that I used up my battery. Anyway, as usual I'll update this post gradually with my thoughts.
The first thing to say tonight is that Survation's Damian Lyons Lowe has not exactly covered himself in glory - he's the boy who cried wolf, and people will quite rightly be sceptical of what he says in future if he ever again attempts to hype up a poll in advance. His suggestion overnight that this poll was "quite something!" could reasonably be interpreted as meaning one of two things - either a) a big swing to Yes, or b) a big swing to No. There is absolutely no way of reconciling it with the reality of what the figures show, which appears to be a very, very small swing to Yes. The comment was nothing more than a tiresome marketing ploy, and one that was plainly intended to mislead. I should add of course that this isn't a bloody game - the long-term future of our country is hanging in the balance, nerves are absolutely shredded to such a point that people are practically making themselves ill, and this kind of attention-seeking stunt from a pollster is really not helping anyone.
As far as I can see, Survation have so far only released a small part of their datasets for this poll. That's left me in difficulty trying to make much sense of what I'm looking at, because normally Survation's datasets include two sets of voting intention figures - one filtered for turnout, and one weighted for turnout - with the latter being the headline numbers. There's only one set given in what's been released so far, and it's not at all clear which one it is. Either way, though, it shows the Yes vote edging up very slightly on the last Survation poll...
Should Scotland be an independent country?
Yes 47.1% (+0.5 or +0.2)
No 52.9% (-0.5 or -0.2)
But whether those can be definitively regarded as the headline figures is beyond me at this stage.
As always with Survation, the devil is in the data. There are two fundamental and ongoing problems with their Scottish panel that they seem completely unable or unwilling to resolve - they have far too few people under the age of 25, and far too few people who live in the South of Scotland electoral region. The practical effect is that in poll after poll, they have to upweight the small number of respondents they do have in those two demographic groups by a huge amount, thus magnifying the effect of the random sampling variation that is bound to occur when you are dealing with such a small number of respondents. In theory, that should lead to a greater amount of volatility. Oddly enough, that hasn't happened (with the big exception of the phantom 'No bounce' after the first debate that no other firm detected), but what it certainly leads to is reduced reliability. A claimed margin of error of 3% is fairly meaningless when this kind of extreme upweighting is going on.
Regular readers will probably remember that this was precisely the reason why I was a touch dubious about all of the first three Survation polls that put Yes on 47%. The first two had particularly favourable figures for Yes among the tiny sample of under-25s, who had been upweighted roughly three-fold, while the third poll saw implausibly wonderful figures for Yes among the South of Scotland sample, which had been upweighted two-fold. So when Yes merely "recovered" to 47% in the first Survation poll to be conducted after the second debate, I wondered if it was in real terms the best ever Yes showing with the firm, because it was the first time 47% had been reached with normal-looking figures among young people and in the south. Tonight's poll takes that trend a bit further, because this time Yes have stayed on 47% in spite of figures for young people that are positively dreadful. Of the tiny sample of 57 respondents under the age of 25, no fewer than 51% are voting No and only 32% are voting Yes. Those 57 people have been upweighted to count as 130 'virtual' respondents, meaning that Survation's weighting procedure has created approximately 37 young No voters out of thin air. I know of no good reason to think that there is a 19-point No lead among under-25s (both the YouGov and TNS polls showed an outright Yes lead in that age group), so this looks like a pretty clear-cut case of sampling variation leading to an underestimation of the Yes vote. And that hasn't been offset in any way by the results from the heavily upweighted sample of voters from the south - in fact the No lead in that sample has grown even more since the last poll.
What I've given above is a reason for slightly distrusting both the headline numbers in this poll and the trend. There's also an additional reason for distrusting Survation's headline numbers, but which doesn't have any relevance to the trend they show, and that's their continuing failure to join the new orthodoxy of weighting by country of birth. We don't actually know for a fact that Survation have too many English-born respondents in their samples, but given that it's a problem that has been clearly established among all of the other three online pollsters, it doesn't seem far-fetched to think that it might apply to Survation as well. If it does, an adjustment to correct the error might well reduce the No lead by 2% or so. That may not seem like much, but in the light of the potential closeness of this contest, it's absolutely crucial, and the lack of any adjustment may be distorting our perception of the state of play. Incidentally, that observation may well also apply to the poll expected this coming weekend from ICM, who have similarly refused so far to introduce country of birth weighting in spite of having already established that they have considerably too many English-born respondents in their samples.
Last night, I stumbled upon someone live-tweeting a lecture from Professor Curtice, who was apparently saying that "something" was definitely happening, but it was unclear whether that something was a swing, or convergence between pollsters - and that the Survation poll would help to solve the puzzle. Presumably the verdict is now going to be convergence, but surely it's self-evident that's a false choice. Convergence couldn't possibly be happening unless a genuine swing had taken place. If the reason for the disagreements about the recent trend is that different pollsters have been measuring something slightly different to each other, that would suggest a big swing has occurred among the type of people TNS and YouGov interview, but not among the type of people Panelbase and Survation interview (who were proportionately more likely to be Yes voters in the first place). Therefore, the most plausible conclusion to draw is that the position on the ground is somewhere in the middle - there has been a swing to Yes, but not quite as big a one as suggested by YouGov and TNS.
However, there's a caveat about the whole convergence theory, because Survation's contribution to it depends on the assumption that the new poll is directly comparable to the TNS and YouGov polls, which as a result of the fieldwork dates is not fully the case. Survation's fieldwork can be split into two distinct periods - Friday and Saturday, before the pandemonium started when YouGov gave Yes a slight lead, and Sunday to Tuesday. There are three possibilities of what has happened -
1) There was a swing to Yes in Survation's fieldwork on Friday and Saturday, in line with what YouGov and TNS showed. That would mean there must have been a significant drop in Yes support from Sunday onwards in order for the results to average out as unchanged. If so, we can expect a significant swing to No in tomorrow night's YouGov poll.
2) There was an absolutely steady picture across the five days of Survation's fieldwork. That would be wholly consistent with Curtice's convergence theory, but paradoxically would mean that all bets are off for tomorrow night's YouGov. We've got used to the headline numbers of different firms diverging in this campaign, but until now the trends have often been broadly similar - if that's no longer going to be the case, then predictions are utterly impossible.
3) There was actually a fall in Yes support in the fieldwork from Friday and Saturday, which has been offset by a Yes bandwagon effect from Sunday onwards. This is the least likely of the three possibilities (because evidence from other pollsters contradicts the idea that Yes were slipping prior to Sunday), but the problems with Survation's sampling means that it shouldn't be dismissed out of hand. And if that is what's happened, we might expect a swing to Yes in the YouGov poll.
That said, you'd probably expect some kind of reduction in the Yes vote in the YouGov poll, if only due to a reversion to the mean after the almost unbelievably big swing to Yes that has occurred over the last month. The greater the reduction is, the less likely it is to be caused purely by margin of error noise - although even slippage to 45% or 46% could be just about explained away by sampling variation.
* * *
REQUIRED SWINGS
Swing required for 1 out of 6 pollsters to show Yes ahead or level : 0.0%
Swing required for 2 out of 6 pollsters to show Yes ahead or level : 0.5%
Swing required for 3 out of 6 pollsters to show Yes ahead or level : 2.0%
Swing required for 4 out of 6 pollsters to show Yes ahead or level : 3.0%
Swing required for 5 out of 6 pollsters to show Yes ahead or level : 4.5%
Swing required for 6 out of 6 pollsters to show Yes ahead or level : 7.0%
* * *
SCOT GOES POP POLL OF POLLS
MEAN AVERAGE (excluding Don't Knows) :
Yes 47.0% (n/c)
No 53.0% (n/c)
MEAN AVERAGE (not excluding Don't Knows) :
Yes 41.5% (n/c)
No 46.8% (n/c)
MEDIAN AVERAGE (excluding Don't Knows) :
Yes 47.2% (n/c)
No 52.8% (n/c)
(The Poll of Polls is based on a rolling average of the most recent poll from each of the pollsters that have been active in the referendum campaign since September 2013, and that adhere to British Polling Council rules. At present, there are six - YouGov, TNS-BMRB, Survation, Panelbase, Ipsos-Mori and ICM. Whenever a new poll is published, it replaces the last poll from the same company in the sample. Changes in the Poll of Polls are generally glacial in nature due to the fact that only a small portion of the sample is updated each time.)
The first thing to say tonight is that Survation's Damian Lyons Lowe has not exactly covered himself in glory - he's the boy who cried wolf, and people will quite rightly be sceptical of what he says in future if he ever again attempts to hype up a poll in advance. His suggestion overnight that this poll was "quite something!" could reasonably be interpreted as meaning one of two things - either a) a big swing to Yes, or b) a big swing to No. There is absolutely no way of reconciling it with the reality of what the figures show, which appears to be a very, very small swing to Yes. The comment was nothing more than a tiresome marketing ploy, and one that was plainly intended to mislead. I should add of course that this isn't a bloody game - the long-term future of our country is hanging in the balance, nerves are absolutely shredded to such a point that people are practically making themselves ill, and this kind of attention-seeking stunt from a pollster is really not helping anyone.
As far as I can see, Survation have so far only released a small part of their datasets for this poll. That's left me in difficulty trying to make much sense of what I'm looking at, because normally Survation's datasets include two sets of voting intention figures - one filtered for turnout, and one weighted for turnout - with the latter being the headline numbers. There's only one set given in what's been released so far, and it's not at all clear which one it is. Either way, though, it shows the Yes vote edging up very slightly on the last Survation poll...
Should Scotland be an independent country?
Yes 47.1% (+0.5 or +0.2)
No 52.9% (-0.5 or -0.2)
But whether those can be definitively regarded as the headline figures is beyond me at this stage.
As always with Survation, the devil is in the data. There are two fundamental and ongoing problems with their Scottish panel that they seem completely unable or unwilling to resolve - they have far too few people under the age of 25, and far too few people who live in the South of Scotland electoral region. The practical effect is that in poll after poll, they have to upweight the small number of respondents they do have in those two demographic groups by a huge amount, thus magnifying the effect of the random sampling variation that is bound to occur when you are dealing with such a small number of respondents. In theory, that should lead to a greater amount of volatility. Oddly enough, that hasn't happened (with the big exception of the phantom 'No bounce' after the first debate that no other firm detected), but what it certainly leads to is reduced reliability. A claimed margin of error of 3% is fairly meaningless when this kind of extreme upweighting is going on.
Regular readers will probably remember that this was precisely the reason why I was a touch dubious about all of the first three Survation polls that put Yes on 47%. The first two had particularly favourable figures for Yes among the tiny sample of under-25s, who had been upweighted roughly three-fold, while the third poll saw implausibly wonderful figures for Yes among the South of Scotland sample, which had been upweighted two-fold. So when Yes merely "recovered" to 47% in the first Survation poll to be conducted after the second debate, I wondered if it was in real terms the best ever Yes showing with the firm, because it was the first time 47% had been reached with normal-looking figures among young people and in the south. Tonight's poll takes that trend a bit further, because this time Yes have stayed on 47% in spite of figures for young people that are positively dreadful. Of the tiny sample of 57 respondents under the age of 25, no fewer than 51% are voting No and only 32% are voting Yes. Those 57 people have been upweighted to count as 130 'virtual' respondents, meaning that Survation's weighting procedure has created approximately 37 young No voters out of thin air. I know of no good reason to think that there is a 19-point No lead among under-25s (both the YouGov and TNS polls showed an outright Yes lead in that age group), so this looks like a pretty clear-cut case of sampling variation leading to an underestimation of the Yes vote. And that hasn't been offset in any way by the results from the heavily upweighted sample of voters from the south - in fact the No lead in that sample has grown even more since the last poll.
What I've given above is a reason for slightly distrusting both the headline numbers in this poll and the trend. There's also an additional reason for distrusting Survation's headline numbers, but which doesn't have any relevance to the trend they show, and that's their continuing failure to join the new orthodoxy of weighting by country of birth. We don't actually know for a fact that Survation have too many English-born respondents in their samples, but given that it's a problem that has been clearly established among all of the other three online pollsters, it doesn't seem far-fetched to think that it might apply to Survation as well. If it does, an adjustment to correct the error might well reduce the No lead by 2% or so. That may not seem like much, but in the light of the potential closeness of this contest, it's absolutely crucial, and the lack of any adjustment may be distorting our perception of the state of play. Incidentally, that observation may well also apply to the poll expected this coming weekend from ICM, who have similarly refused so far to introduce country of birth weighting in spite of having already established that they have considerably too many English-born respondents in their samples.
Last night, I stumbled upon someone live-tweeting a lecture from Professor Curtice, who was apparently saying that "something" was definitely happening, but it was unclear whether that something was a swing, or convergence between pollsters - and that the Survation poll would help to solve the puzzle. Presumably the verdict is now going to be convergence, but surely it's self-evident that's a false choice. Convergence couldn't possibly be happening unless a genuine swing had taken place. If the reason for the disagreements about the recent trend is that different pollsters have been measuring something slightly different to each other, that would suggest a big swing has occurred among the type of people TNS and YouGov interview, but not among the type of people Panelbase and Survation interview (who were proportionately more likely to be Yes voters in the first place). Therefore, the most plausible conclusion to draw is that the position on the ground is somewhere in the middle - there has been a swing to Yes, but not quite as big a one as suggested by YouGov and TNS.
However, there's a caveat about the whole convergence theory, because Survation's contribution to it depends on the assumption that the new poll is directly comparable to the TNS and YouGov polls, which as a result of the fieldwork dates is not fully the case. Survation's fieldwork can be split into two distinct periods - Friday and Saturday, before the pandemonium started when YouGov gave Yes a slight lead, and Sunday to Tuesday. There are three possibilities of what has happened -
1) There was a swing to Yes in Survation's fieldwork on Friday and Saturday, in line with what YouGov and TNS showed. That would mean there must have been a significant drop in Yes support from Sunday onwards in order for the results to average out as unchanged. If so, we can expect a significant swing to No in tomorrow night's YouGov poll.
2) There was an absolutely steady picture across the five days of Survation's fieldwork. That would be wholly consistent with Curtice's convergence theory, but paradoxically would mean that all bets are off for tomorrow night's YouGov. We've got used to the headline numbers of different firms diverging in this campaign, but until now the trends have often been broadly similar - if that's no longer going to be the case, then predictions are utterly impossible.
3) There was actually a fall in Yes support in the fieldwork from Friday and Saturday, which has been offset by a Yes bandwagon effect from Sunday onwards. This is the least likely of the three possibilities (because evidence from other pollsters contradicts the idea that Yes were slipping prior to Sunday), but the problems with Survation's sampling means that it shouldn't be dismissed out of hand. And if that is what's happened, we might expect a swing to Yes in the YouGov poll.
That said, you'd probably expect some kind of reduction in the Yes vote in the YouGov poll, if only due to a reversion to the mean after the almost unbelievably big swing to Yes that has occurred over the last month. The greater the reduction is, the less likely it is to be caused purely by margin of error noise - although even slippage to 45% or 46% could be just about explained away by sampling variation.
* * *
REQUIRED SWINGS
Swing required for 1 out of 6 pollsters to show Yes ahead or level : 0.0%
Swing required for 2 out of 6 pollsters to show Yes ahead or level : 0.5%
Swing required for 3 out of 6 pollsters to show Yes ahead or level : 2.0%
Swing required for 4 out of 6 pollsters to show Yes ahead or level : 3.0%
Swing required for 5 out of 6 pollsters to show Yes ahead or level : 4.5%
Swing required for 6 out of 6 pollsters to show Yes ahead or level : 7.0%
* * *
SCOT GOES POP POLL OF POLLS
MEAN AVERAGE (excluding Don't Knows) :
Yes 47.0% (n/c)
No 53.0% (n/c)
MEAN AVERAGE (not excluding Don't Knows) :
Yes 41.5% (n/c)
No 46.8% (n/c)
MEDIAN AVERAGE (excluding Don't Knows) :
Yes 47.2% (n/c)
No 52.8% (n/c)
(The Poll of Polls is based on a rolling average of the most recent poll from each of the pollsters that have been active in the referendum campaign since September 2013, and that adhere to British Polling Council rules. At present, there are six - YouGov, TNS-BMRB, Survation, Panelbase, Ipsos-Mori and ICM. Whenever a new poll is published, it replaces the last poll from the same company in the sample. Changes in the Poll of Polls are generally glacial in nature due to the fact that only a small portion of the sample is updated each time.)
Wednesday, September 10, 2014
Yes campaign remain on all-time high in new Survation poll
New Survation poll :
Should Scotland be an independent country?
Yes 47% (n/c)
No 53% (n/c)
UPDATE : I'm now adding some analysis and a Poll of Polls update in a fresh post HERE.
Should Scotland be an independent country?
Yes 47% (n/c)
No 53% (n/c)
UPDATE : I'm now adding some analysis and a Poll of Polls update in a fresh post HERE.
An important point about tonight's Survation poll
A new Survation poll is due to be released at 10.30 this evening, although there's always a chance the embargo will be broken earlier. The firm's chief has tweeted that the results are "quite something". That could mean something good or bad for us, so I'm not going to speculate (unless anyone has heard something from a semi-credible source).
But there is something important that needs to be clarified. It's not actually true that this poll will be the first to be wholly conducted since the public realised that Yes were in with a real chance of winning. In fact, the fieldwork took place between Friday and yesterday, which covers a span of five days, two of which were before the news broke about Sunday's YouGov poll. Indeed, to a small extent the fieldwork even overlapped with the fieldwork for the YouGov poll. I don't know if Survation have procedures in place to spread out their interview invitations over the course of the fieldwork period, but if not, it's quite possible that MOST of this poll will have been conducted before the YouGov news broke, because most participants tend to respond straight away.
So whatever the poll shows, we should certainly bear that in mind when interpreting the results.
Wisdom on Wednesday : Is there even one example?
"'Let's go back to being governed by our larger neighbour,' said no independent nation ever."
Novelist and ex-journalist Christopher Brookmyre.
Novelist and ex-journalist Christopher Brookmyre.
Tuesday, September 9, 2014
Nigel's back!
A guest post by Robert MacDonald
On 12 September, the Farage bandwagon will roll into town. "There’s been too much bean-counting on one side and all the passion has been on the Yes side. It’s time to change that," a UKIP spokesman was reported to have said in the Sunday Post. According to the BBC, "We take a position that we will not be intimidated off the streets, and if Jim [Murphy] wants any support on his speaking tour when he goes back out there we are happy to join him to make sure he stays safe."
Overegged claims of intimidation aside, it is clear who will benefit most from such a circus. It won't be Jim Murphy, who would be completely overshadowed by his UKIP minders. It won't be the people of Scotland. It won't even be the union. It will be Nigel Farage himself, his image as a plain-speaking defender of the English everyman beamed into every home receiving the BBC, the new enemy the racist Scot, opening deep fissures in the very union he claims to uphold. Another little nugget of hate will have been lodged in the minds of people who have been let down by their politicians, manipulated by their media, people who look around and wonder whose hand will help them out of their current state of impotent fear. Ed Milliband’s? David Cameron’s?
Let's face it, people across the whole of the UK are scunnered with politicians, angry at their own personal helplessness at the way the place is run. UKIP tempts them with its easy scapegoats. UKIP is the only visible alternative. But UKIP isn't the answer. Voting UKIP is a cry for help. Voting UKIP is the electorate slashing its wrists in despair. UKIP's policies are the most reactionary of any party out there and once in power, would only cement the position of the elite - which may explain why the media have been all over Nigel Farage like a cheap suit for years. Voting UKIP is a vote to make things worse for the vast majority.
In Scotland though, a window of opportunity has opened up. People have started talking about the sort of country they want to live in. It's infectious. And it is now beyond the politicians' control. When No voters talk about Alex Salmond, they are behind the curve. The SNP started this process, but the Yes grassroots have become their own force. If you aren't involved in it you wouldn't know about it as the media don't discuss it. Sites like this are Scotland’s open secret. Most of the rest of the UK is profoundly unaware. Many in Scotland too. But it's happening, on the internet, in conversation after conversation, in doorstep meetings and packed town halls. The ugliness of UKIP is nowhere to be seen for one good reason. People aren’t lashing out, looking for something to blame. They are instead imagining a blank slate, testing their own mettle for change, thirsting for responsibility for their own lives. But the window for discovery is brief. On 18 September, it closes.
I am fascinated to see where this goes. Scotland votes Yes and briefly, all things become possible. But who will end up in charge - the elite and their politicians again? Will Yes campaigners, an independent Scotland delivered, relax and think the job done when really, the only change will have been the permission to continue the conversation, this sweeping away of apathy, this transformational blossoming of trust, responsibility and opportunity in the people themselves? Equally, should Scotland vote No - will the people who have dared to discuss how they want their society organised shut up? It is hard to see how. Either way, the independence referendum has created a breach in the way we do and understand politics, letting in a ray of democratic light. Fear not to speak your own aspirations – this conversation is a virus, your voice its carrier. Should enough catch it, Yes will be inevitable.
On 12 September, the Farage bandwagon will roll into town. "There’s been too much bean-counting on one side and all the passion has been on the Yes side. It’s time to change that," a UKIP spokesman was reported to have said in the Sunday Post. According to the BBC, "We take a position that we will not be intimidated off the streets, and if Jim [Murphy] wants any support on his speaking tour when he goes back out there we are happy to join him to make sure he stays safe."
Overegged claims of intimidation aside, it is clear who will benefit most from such a circus. It won't be Jim Murphy, who would be completely overshadowed by his UKIP minders. It won't be the people of Scotland. It won't even be the union. It will be Nigel Farage himself, his image as a plain-speaking defender of the English everyman beamed into every home receiving the BBC, the new enemy the racist Scot, opening deep fissures in the very union he claims to uphold. Another little nugget of hate will have been lodged in the minds of people who have been let down by their politicians, manipulated by their media, people who look around and wonder whose hand will help them out of their current state of impotent fear. Ed Milliband’s? David Cameron’s?
Let's face it, people across the whole of the UK are scunnered with politicians, angry at their own personal helplessness at the way the place is run. UKIP tempts them with its easy scapegoats. UKIP is the only visible alternative. But UKIP isn't the answer. Voting UKIP is a cry for help. Voting UKIP is the electorate slashing its wrists in despair. UKIP's policies are the most reactionary of any party out there and once in power, would only cement the position of the elite - which may explain why the media have been all over Nigel Farage like a cheap suit for years. Voting UKIP is a vote to make things worse for the vast majority.
In Scotland though, a window of opportunity has opened up. People have started talking about the sort of country they want to live in. It's infectious. And it is now beyond the politicians' control. When No voters talk about Alex Salmond, they are behind the curve. The SNP started this process, but the Yes grassroots have become their own force. If you aren't involved in it you wouldn't know about it as the media don't discuss it. Sites like this are Scotland’s open secret. Most of the rest of the UK is profoundly unaware. Many in Scotland too. But it's happening, on the internet, in conversation after conversation, in doorstep meetings and packed town halls. The ugliness of UKIP is nowhere to be seen for one good reason. People aren’t lashing out, looking for something to blame. They are instead imagining a blank slate, testing their own mettle for change, thirsting for responsibility for their own lives. But the window for discovery is brief. On 18 September, it closes.
I am fascinated to see where this goes. Scotland votes Yes and briefly, all things become possible. But who will end up in charge - the elite and their politicians again? Will Yes campaigners, an independent Scotland delivered, relax and think the job done when really, the only change will have been the permission to continue the conversation, this sweeping away of apathy, this transformational blossoming of trust, responsibility and opportunity in the people themselves? Equally, should Scotland vote No - will the people who have dared to discuss how they want their society organised shut up? It is hard to see how. Either way, the independence referendum has created a breach in the way we do and understand politics, letting in a ray of democratic light. Fear not to speak your own aspirations – this conversation is a virus, your voice its carrier. Should enough catch it, Yes will be inevitable.
Monday, September 8, 2014
Yes campaign draw level in electrifying poll from traditionally No-friendly firm TNS-BMRB
Hmmm. I think at some point over the last few weeks we must have had that "game-changer" that the London media kept telling us we needed. Did you get the impression they weren't really expecting it to happen, though? Here is the extraordinary latest referendum poll from TNS-BMRB...
Should Scotland be an independent country?
Yes 50% (+8)
No 50% (-8)
As you'll have seen from the previous post, I wasn't expecting a swing on anything like that scale. However, there's a very straightforward explanation for this surprise, which is that TNS-BMRB's fieldwork is nowhere near as far out of date as it usually is. It started less than two weeks ago on the 27th of August, meaning that it took place entirely after the second leaders' debate on the 25th. The other crucial significance of the 27th is, of course, that it was the first date on which people can realistically be expected to have started voting by post. So we now have quite a significant body of evidence that, at the very least, No were not that far ahead when the referendum got underway. And quite possibly, the contest was more or less even-stevens.
There's almost a sense of TNS-BMRB "coming home" with this poll, because although they've become known during the referendum campaign as one of the three No-friendly pollsters (alongside YouGov and Ipsos-Mori), it wasn't all that long ago that they were by far the favourite firm of independence supporters everywhere. As recently as the late summer of 2011 they showed a small lead for Yes, which was hardly untypical for them - they had shown the odd plurality for independence before that, and even when they had the anti-independence vote ahead it usually wasn't by much. Something very peculiar happened in 2012, though, and TNS moved suddenly and decisively towards the No-friendly end of the spectrum - exactly why that occurred is one of the many, many baffling mysteries about polling during this campaign. And yet here they are, back again as (at least temporarily) a more Yes-friendly pollster than even Panelbase.
This poll is obviously the first strong piece of evidence that the massive-scale swing to Yes is not something that is going to be confined solely to YouGov. However, given the inexplicable extent of the disagreement between YouGov and Panelbase about the recent trend, the TNS figures still can't be taken as proof that we are going to see this hugely encouraging pattern across the board. It seems to me there are now two plausible possibilities - either a) there is going to be a straight split down the middle among the polling firms, with ICM, Survation and Panelbase (all of whom have relatively similar methodologies) not showing as big a movement towards Yes as the others, or b) Panelbase will prove to be the outright odd-one-out, with all the others showing a TNS/YouGov-type swing. The latter possibility would constitute a strange turn of events, but far from an unprecedented one - during the winter Panelbase bamboozled us all by producing a swing to No (albeit within the margin of error) at a time when there was otherwise a broad consensus that Yes were making significant progress. It could be that there's something a bit unusual about Panelbase's sample - maybe they've been overpolled, or maybe they were just especially politically committed from the outset, and are thus less likely to be open to persuasion than voters out there in the real world. It's certainly striking how stable Panelbase's numbers have been throughout this campaign - yes, they've shown a modest drift towards Yes this year, but nothing on the scale that other firms have reported. (And that was the case even before the drama of recent events.)
Let's stick with the point about 'the real world', because that's what makes both TNS-BMRB and Ipsos-Mori special in this campaign - in their own very different ways, they actively seek out a completely fresh sample for every poll. If it wasn't for them, we'd be completely reliant on pollsters who use volunteer online panels, with all the danger that entails that the people being interviewed are not truly representative of the electorate at large. Until now, it's been a matter of considerable concern that the only two non-online companies were so firmly on the No-friendly end of the spectrum, so a telling breakthrough for Yes in a TNS poll effectively counts double. Indeed, because of what Panelbase are showing, we can't completely exclude the possibility that we've now swung to the other extreme, and that in the closing days of the campaign 'real world' polls (assuming there are more to come) will show a slightly more favourable position for Yes than the volunteer online panels do. Admittedly I don't think that's at all likely, but it's no longer inconceivable.
Of course there's still room for doubt over whether a real world pollster will always produce representative results - for example, Ipsos-Mori might be missing crucial sections of the population due to their apparent practice of only phoning landlines. That isn't a concern for TNS who go out and knock on people's doors, but there's still the issue of how they weight their raw data. It's striking that there's never a big enough gap in their raw sample between people who recall voting SNP in 2011 and people who recall voting Labour. Although there's evidence from other pollsters that people are much better at accurately recalling how they voted in 2011 than how they voted in 2010, it's hard to escape the worry that there might be at least a modest degree of false recall at play among the TNS respondents, in which case the adjustment made to bring the sample into line with the actual 2011 result may be artificially flattering the Yes vote. Fortunately, however, the adjustment in tonight's poll isn't as big as in one or two previous TNS polls, so even if Yes are being overstated due to this issue, it won't be by much.
And we also have to bear in mind the crucial weighting that TNS are failing to apply - namely country of birth. It's mainly among online pollsters that we have evidence that there is almost always too great a number of English-born respondents in the raw data, but if by any chance this general rule also applies to non-online firms, then a weighting to correct the error would probably be sufficient to push Yes into a clear lead in tonight's poll.
Although Panelbase may be in disagreement with TNS and YouGov about the overall trend, the one thing all three firms agree on is that the gender gap has narrowed of late. Even a consensus of three polls doesn't represent proof that it has actually happened, but it's certainly beginning to look very much like a real and potentially highly significant pattern.
The No camp still have the tiniest of leads in this poll on the unrounded figures, and indeed they have a lead even on the rounded figures when Don't Knows are taken into account (39% to 38%). But as always with TNS, it's possible to apply various different turnout filters to the voting intention numbers, and there's one filter which actually puts Yes into the lead...
Whole sample (equivalent to 100% turnout) :
Yes 49.6% (+8.0)
No 50.4% (-8.0)
Whole sample excluding only definite non-voters (equivalent to 97% turnout) :
Yes 49.7% (+6.8)
No 50.3% (-6.8)
Respondents who say they are certain or very likely to vote (equivalent to 91% turnout) :
Yes 50.2% (+5.6)
No 49.8% (-5.6)
Respondents who say they are certain to vote (equivalent to 84% turnout) :
Yes 49.9% (+4.7)
No 50.1% (-4.7)
For the first time with TNS, it's the figures for definite voters that look most likely to correlate with the actual turnout. But given the huge jump in people's reported likelihood to vote in this poll, you do begin to wonder if a 90%+ figure is just about possible. After all, it happened in Quebec...
* * *
REQUIRED SWINGS
Swing required for 1 out of 6 pollsters to show Yes ahead or level : 0.0%
Swing required for 2 out of 6 pollsters to show Yes ahead or level : 0.5%
Swing required for 3 out of 6 pollsters to show Yes ahead or level : 2.0%
Swing required for 4 out of 6 pollsters to show Yes ahead or level : 3.0%
Swing required for 5 out of 6 pollsters to show Yes ahead or level : 4.5%
Swing required for 6 out of 6 pollsters to show Yes ahead or level : 7.0%
* * *
SCOT GOES POP POLL OF POLLS
It's getting to the stage where I have to put a significant health warning on the Poll of Polls. As it now seems likely that there has been a substantial swing since the second leaders' debate, the fact that a full third of the Poll of Polls sample is made up of polls conducted before that debate (Ipsos-Mori and ICM) means that the numbers listed below are likely to be underestimating the true strength of the Yes vote.
MEAN AVERAGE (excluding Don't Knows) :
Yes 47.0% (+1.1)
No 53.0% (-1.1)
MEAN AVERAGE (not excluding Don't Knows) :
Yes 41.5% (+1.0)
No 46.8% (-1.0)
MEDIAN AVERAGE (excluding Don't Knows) :
Yes 47.2% (+1.5)
No 52.8% (-1.5)
(The Poll of Polls is based on a rolling average of the most recent poll from each of the pollsters that have been active in the referendum campaign since September 2013, and that adhere to British Polling Council rules. At present, there are six - YouGov, TNS-BMRB, Survation, Panelbase, Ipsos-Mori and ICM. Whenever a new poll is published, it replaces the last poll from the same company in the sample. Changes in the Poll of Polls are generally glacial in nature due to the fact that only a small portion of the sample is updated each time.)
Should Scotland be an independent country?
Yes 50% (+8)
No 50% (-8)
As you'll have seen from the previous post, I wasn't expecting a swing on anything like that scale. However, there's a very straightforward explanation for this surprise, which is that TNS-BMRB's fieldwork is nowhere near as far out of date as it usually is. It started less than two weeks ago on the 27th of August, meaning that it took place entirely after the second leaders' debate on the 25th. The other crucial significance of the 27th is, of course, that it was the first date on which people can realistically be expected to have started voting by post. So we now have quite a significant body of evidence that, at the very least, No were not that far ahead when the referendum got underway. And quite possibly, the contest was more or less even-stevens.
There's almost a sense of TNS-BMRB "coming home" with this poll, because although they've become known during the referendum campaign as one of the three No-friendly pollsters (alongside YouGov and Ipsos-Mori), it wasn't all that long ago that they were by far the favourite firm of independence supporters everywhere. As recently as the late summer of 2011 they showed a small lead for Yes, which was hardly untypical for them - they had shown the odd plurality for independence before that, and even when they had the anti-independence vote ahead it usually wasn't by much. Something very peculiar happened in 2012, though, and TNS moved suddenly and decisively towards the No-friendly end of the spectrum - exactly why that occurred is one of the many, many baffling mysteries about polling during this campaign. And yet here they are, back again as (at least temporarily) a more Yes-friendly pollster than even Panelbase.
This poll is obviously the first strong piece of evidence that the massive-scale swing to Yes is not something that is going to be confined solely to YouGov. However, given the inexplicable extent of the disagreement between YouGov and Panelbase about the recent trend, the TNS figures still can't be taken as proof that we are going to see this hugely encouraging pattern across the board. It seems to me there are now two plausible possibilities - either a) there is going to be a straight split down the middle among the polling firms, with ICM, Survation and Panelbase (all of whom have relatively similar methodologies) not showing as big a movement towards Yes as the others, or b) Panelbase will prove to be the outright odd-one-out, with all the others showing a TNS/YouGov-type swing. The latter possibility would constitute a strange turn of events, but far from an unprecedented one - during the winter Panelbase bamboozled us all by producing a swing to No (albeit within the margin of error) at a time when there was otherwise a broad consensus that Yes were making significant progress. It could be that there's something a bit unusual about Panelbase's sample - maybe they've been overpolled, or maybe they were just especially politically committed from the outset, and are thus less likely to be open to persuasion than voters out there in the real world. It's certainly striking how stable Panelbase's numbers have been throughout this campaign - yes, they've shown a modest drift towards Yes this year, but nothing on the scale that other firms have reported. (And that was the case even before the drama of recent events.)
Let's stick with the point about 'the real world', because that's what makes both TNS-BMRB and Ipsos-Mori special in this campaign - in their own very different ways, they actively seek out a completely fresh sample for every poll. If it wasn't for them, we'd be completely reliant on pollsters who use volunteer online panels, with all the danger that entails that the people being interviewed are not truly representative of the electorate at large. Until now, it's been a matter of considerable concern that the only two non-online companies were so firmly on the No-friendly end of the spectrum, so a telling breakthrough for Yes in a TNS poll effectively counts double. Indeed, because of what Panelbase are showing, we can't completely exclude the possibility that we've now swung to the other extreme, and that in the closing days of the campaign 'real world' polls (assuming there are more to come) will show a slightly more favourable position for Yes than the volunteer online panels do. Admittedly I don't think that's at all likely, but it's no longer inconceivable.
Of course there's still room for doubt over whether a real world pollster will always produce representative results - for example, Ipsos-Mori might be missing crucial sections of the population due to their apparent practice of only phoning landlines. That isn't a concern for TNS who go out and knock on people's doors, but there's still the issue of how they weight their raw data. It's striking that there's never a big enough gap in their raw sample between people who recall voting SNP in 2011 and people who recall voting Labour. Although there's evidence from other pollsters that people are much better at accurately recalling how they voted in 2011 than how they voted in 2010, it's hard to escape the worry that there might be at least a modest degree of false recall at play among the TNS respondents, in which case the adjustment made to bring the sample into line with the actual 2011 result may be artificially flattering the Yes vote. Fortunately, however, the adjustment in tonight's poll isn't as big as in one or two previous TNS polls, so even if Yes are being overstated due to this issue, it won't be by much.
And we also have to bear in mind the crucial weighting that TNS are failing to apply - namely country of birth. It's mainly among online pollsters that we have evidence that there is almost always too great a number of English-born respondents in the raw data, but if by any chance this general rule also applies to non-online firms, then a weighting to correct the error would probably be sufficient to push Yes into a clear lead in tonight's poll.
Although Panelbase may be in disagreement with TNS and YouGov about the overall trend, the one thing all three firms agree on is that the gender gap has narrowed of late. Even a consensus of three polls doesn't represent proof that it has actually happened, but it's certainly beginning to look very much like a real and potentially highly significant pattern.
The No camp still have the tiniest of leads in this poll on the unrounded figures, and indeed they have a lead even on the rounded figures when Don't Knows are taken into account (39% to 38%). But as always with TNS, it's possible to apply various different turnout filters to the voting intention numbers, and there's one filter which actually puts Yes into the lead...
Whole sample (equivalent to 100% turnout) :
Yes 49.6% (+8.0)
No 50.4% (-8.0)
Whole sample excluding only definite non-voters (equivalent to 97% turnout) :
Yes 49.7% (+6.8)
No 50.3% (-6.8)
Respondents who say they are certain or very likely to vote (equivalent to 91% turnout) :
Yes 50.2% (+5.6)
No 49.8% (-5.6)
Respondents who say they are certain to vote (equivalent to 84% turnout) :
Yes 49.9% (+4.7)
No 50.1% (-4.7)
For the first time with TNS, it's the figures for definite voters that look most likely to correlate with the actual turnout. But given the huge jump in people's reported likelihood to vote in this poll, you do begin to wonder if a 90%+ figure is just about possible. After all, it happened in Quebec...
* * *
REQUIRED SWINGS
Swing required for 1 out of 6 pollsters to show Yes ahead or level : 0.0%
Swing required for 2 out of 6 pollsters to show Yes ahead or level : 0.5%
Swing required for 3 out of 6 pollsters to show Yes ahead or level : 2.0%
Swing required for 4 out of 6 pollsters to show Yes ahead or level : 3.0%
Swing required for 5 out of 6 pollsters to show Yes ahead or level : 4.5%
Swing required for 6 out of 6 pollsters to show Yes ahead or level : 7.0%
* * *
SCOT GOES POP POLL OF POLLS
It's getting to the stage where I have to put a significant health warning on the Poll of Polls. As it now seems likely that there has been a substantial swing since the second leaders' debate, the fact that a full third of the Poll of Polls sample is made up of polls conducted before that debate (Ipsos-Mori and ICM) means that the numbers listed below are likely to be underestimating the true strength of the Yes vote.
MEAN AVERAGE (excluding Don't Knows) :
Yes 47.0% (+1.1)
No 53.0% (-1.1)
MEAN AVERAGE (not excluding Don't Knows) :
Yes 41.5% (+1.0)
No 46.8% (-1.0)
MEDIAN AVERAGE (excluding Don't Knows) :
Yes 47.2% (+1.5)
No 52.8% (-1.5)
(The Poll of Polls is based on a rolling average of the most recent poll from each of the pollsters that have been active in the referendum campaign since September 2013, and that adhere to British Polling Council rules. At present, there are six - YouGov, TNS-BMRB, Survation, Panelbase, Ipsos-Mori and ICM. Whenever a new poll is published, it replaces the last poll from the same company in the sample. Changes in the Poll of Polls are generally glacial in nature due to the fact that only a small portion of the sample is updated each time.)
A primer for the forthcoming TNS-BMRB poll
Apparently the new TNS-BMRB referendum poll, which we were originally told to expect mid-week, has been brought forward. I'm still not sure exactly when to expect it, because there's been no rhyme or reason to when they've released polls in the past - they've occasionally done it at weird times like the middle of the afternoon, but it'll probably be tonight.
Those of you who follow the increasingly confusing utterances of our old friend Mike "can't be arsed" Smithson may have spotted that he's been bigging up this poll to an absurd degree, almost according it Stalingrad-style significance as the poll which will finally settle the dispute over whether YouGov are right (meaning there has been a very dramatic and very recent swing to Yes) or Panelbase are right (meaning there has been relative stability over the last couple of weeks). Let me be clear - that is absolute nonsense. I don't say that because TNS-BMRB aren't a credible pollster, but simply because a large part of their fieldwork will be hopelessly out of date. On past form, they will have started conducting interviews for this poll the best part of a month ago, not all that long after the first debate, when the media were still churning out fairy-tales about a "Darling win". And of the aforementioned points about the swing YouGov are showing, the most important one is that it's very recent - meaning that even if it's 100% real, there's no way that TNS can realistically hope to pick it up.
I've no idea what the trend will be in this poll, but we can safely assume that any movement is likely to be modest, simply because of the dates. The last poll showed Yes on their second-highest level of support of the campaign so far with the No-friendly TNS methodology - they were on 42% and No were on 58%. Admittedly, the turnout-filtered figures were somewhat more Yes-friendly, putting Yes on 45% and No on 55%, but those figures have proved to be much more volatile over recent months.
A cynic might wonder if Smithson understands all of this perfectly well, and is only making a song and dance about TNS because he's already got a bogus "No comeback" narrative up his sleeve. Incidentally, a more interesting point that he's made is that TNS always tack their referendum polling onto their much broader consumer surveys. I wasn't previously aware of that, and it sets a small alarm bell ringing in my mind, because I seem to recall reading many years ago about a polling firm in the 1979 general election that produced far, far higher Conservative leads than anyone else, and were ultimately proved wrong. They couldn't understand where they were going astray, because in principle their methodology was perfectly standard, but the suspicion was that because they were geared as a company towards consumer surveys, unconscious biases in the selection of people to interview were making themselves felt.
UPDATE (4.30pm) : The early hints about the TNS poll are contradicting some of what I said above, with one person in the know apparently describing the results as a "sensation". See the comments section below for more details.
* * *
I think we have a new turbo-charged definition of the word 'fickle' -
London media yesterday : Over the next ten days, NOTHING matters apart from saving the most glorious political union the universe has ever seen. NOTHING.
London media today : Royal baby! Royal baby! Royal baby! Coo! Coo! Kate! Wills! Coo!
* * *
A special message for readers in Wales - a cross-party event in support of a Scottish Yes vote will take place outside the Senedd in Cardiff Bay on Saturday at 2pm. Speakers will include Leanne Wood (leader of Plaid Cymru), Pippa Bartolotti (leader of the Green party in Wales), Ray Davies (a Labour councillor from Caerphilly), Amy Kitcher (a former Liberal Democrat candidate for both Westminster and the Welsh Assembly), Jamie Wallace (from the SNP), and Andrew Redmond Barr (from National Collective). There will be live entertainment at the event, which will be positive and uplifting in nature to counteract the wall of media scare stories.
I must say it's particularly encouraging to see the support that Green politicians in both England and Wales have been giving to the Yes campaign - I initially feared they might prove to be slightly ambivalent.
* * *
I was out and about in a very sunny Glasgow on Saturday, once again sporting my elephant-sized Yes badge, and I happened to notice Bad Romance being performed in St Enoch's Square as I walked past. I was facing the wrong way, so I've no idea if the real Zara Gladman was there in person, but I did wonder if it was entirely wise to be singing a song that goes : "No, no, no, no, no, oh no, gonnae no, oh no, no, no, no, I'm voting NO-OH-OH". That may be just slightly too subtle a pro-Yes message for a random passer-by to fully appreciate!
Later on, I was on a train, and a slightly drunk young woman (I keep meeting them, but this one was Irish) asked me if I'd like to eat her chips for her. I said 'no thanks', and she looked disappointed, but then she saw my Yes badge and her hopes were renewed. "Yes?"
She was still talking about it a few minutes later : "Well, I've done my best to give them away - this guy says Yes, which I agree with him about by the way, but he says No to chips." At that point, my heart finally melted, and I ate a couple of her chips, while listening to her brilliantly explain the referendum to a seven-year-old boy : "Suppose I told you that if you wanted to have crisps on a Friday, you could decide to have crisps on a Friday, would you say Yes to that? Or would you say No, and choose to be part of a bigger group that might force you to eat haddock?"
Those of you who follow the increasingly confusing utterances of our old friend Mike "can't be arsed" Smithson may have spotted that he's been bigging up this poll to an absurd degree, almost according it Stalingrad-style significance as the poll which will finally settle the dispute over whether YouGov are right (meaning there has been a very dramatic and very recent swing to Yes) or Panelbase are right (meaning there has been relative stability over the last couple of weeks). Let me be clear - that is absolute nonsense. I don't say that because TNS-BMRB aren't a credible pollster, but simply because a large part of their fieldwork will be hopelessly out of date. On past form, they will have started conducting interviews for this poll the best part of a month ago, not all that long after the first debate, when the media were still churning out fairy-tales about a "Darling win". And of the aforementioned points about the swing YouGov are showing, the most important one is that it's very recent - meaning that even if it's 100% real, there's no way that TNS can realistically hope to pick it up.
I've no idea what the trend will be in this poll, but we can safely assume that any movement is likely to be modest, simply because of the dates. The last poll showed Yes on their second-highest level of support of the campaign so far with the No-friendly TNS methodology - they were on 42% and No were on 58%. Admittedly, the turnout-filtered figures were somewhat more Yes-friendly, putting Yes on 45% and No on 55%, but those figures have proved to be much more volatile over recent months.
A cynic might wonder if Smithson understands all of this perfectly well, and is only making a song and dance about TNS because he's already got a bogus "No comeback" narrative up his sleeve. Incidentally, a more interesting point that he's made is that TNS always tack their referendum polling onto their much broader consumer surveys. I wasn't previously aware of that, and it sets a small alarm bell ringing in my mind, because I seem to recall reading many years ago about a polling firm in the 1979 general election that produced far, far higher Conservative leads than anyone else, and were ultimately proved wrong. They couldn't understand where they were going astray, because in principle their methodology was perfectly standard, but the suspicion was that because they were geared as a company towards consumer surveys, unconscious biases in the selection of people to interview were making themselves felt.
UPDATE (4.30pm) : The early hints about the TNS poll are contradicting some of what I said above, with one person in the know apparently describing the results as a "sensation". See the comments section below for more details.
* * *
I think we have a new turbo-charged definition of the word 'fickle' -
London media yesterday : Over the next ten days, NOTHING matters apart from saving the most glorious political union the universe has ever seen. NOTHING.
London media today : Royal baby! Royal baby! Royal baby! Coo! Coo! Kate! Wills! Coo!
* * *
A special message for readers in Wales - a cross-party event in support of a Scottish Yes vote will take place outside the Senedd in Cardiff Bay on Saturday at 2pm. Speakers will include Leanne Wood (leader of Plaid Cymru), Pippa Bartolotti (leader of the Green party in Wales), Ray Davies (a Labour councillor from Caerphilly), Amy Kitcher (a former Liberal Democrat candidate for both Westminster and the Welsh Assembly), Jamie Wallace (from the SNP), and Andrew Redmond Barr (from National Collective). There will be live entertainment at the event, which will be positive and uplifting in nature to counteract the wall of media scare stories.
I must say it's particularly encouraging to see the support that Green politicians in both England and Wales have been giving to the Yes campaign - I initially feared they might prove to be slightly ambivalent.
* * *
I was out and about in a very sunny Glasgow on Saturday, once again sporting my elephant-sized Yes badge, and I happened to notice Bad Romance being performed in St Enoch's Square as I walked past. I was facing the wrong way, so I've no idea if the real Zara Gladman was there in person, but I did wonder if it was entirely wise to be singing a song that goes : "No, no, no, no, no, oh no, gonnae no, oh no, no, no, no, I'm voting NO-OH-OH". That may be just slightly too subtle a pro-Yes message for a random passer-by to fully appreciate!
Later on, I was on a train, and a slightly drunk young woman (I keep meeting them, but this one was Irish) asked me if I'd like to eat her chips for her. I said 'no thanks', and she looked disappointed, but then she saw my Yes badge and her hopes were renewed. "Yes?"
She was still talking about it a few minutes later : "Well, I've done my best to give them away - this guy says Yes, which I agree with him about by the way, but he says No to chips." At that point, my heart finally melted, and I ate a couple of her chips, while listening to her brilliantly explain the referendum to a seven-year-old boy : "Suppose I told you that if you wanted to have crisps on a Friday, you could decide to have crisps on a Friday, would you say Yes to that? Or would you say No, and choose to be part of a bigger group that might force you to eat haddock?"
Sunday, September 7, 2014
A wander through the YouGov and Panelbase datasets
I've now had a look at the datasets from both the YouGov and Panelbase polls - the Panelbase datasets aren't on the firm's website yet, but Ivor Knox very kindly sent me a copy. Of course the first thing I always look for is the unrounded voting intention figures (or rather the figures rounded to only one decimal place), and on that front there is good news and bad news...
YouGov :
Yes 51.2% (+4.4)
No 48.8% (-4.4)
That's better than I originally expected, and the reason is that the YouGov poll was actually slightly misreported in many media outlets - the split with Don't Knows included is Yes 47% (+5), No 45% (-3), rather than Yes 47%, No 46%. As a result, I've had to make a slight correction to the latest Poll of Polls update - the average No lead is now a further 0.2% smaller.
The less good news is that Yes only barely made it to 48% in the Panelbase figures, thanks to the effect of rounding -
Panelbase :
Yes 47.5% (-0.1)
No 52.5% (+0.1)
So that further deepens the mystery of why the Panelbase numbers don't appear to be budging at all (albeit from an all-time high position for Yes), at exactly the same time as the No vote appears to be in freefall with the formerly No-friendly firm YouGov. The gender gap has been more or less wiped out in the Panelbase poll - No lead by 53.2% to 46.8% among women and by 51.7% to 48.3% among men, which reverses a modest Yes lead among men in the last poll. It would be tempting to believe that the swing to Yes among women must be real and that the swing to No among men is an illusion, but there's a danger of wishful thinking in that kind of speculation - it's just as possible that normal sampling variation is at play, and that the Yes figure is a bit too high among women and a bit too low among men.
Nevertheless, it's true that if you isolate out the figures for women only, today's two polls are actually identical, because YouGov are also showing figures of Yes 47%, No 53% for female respondents. Moreover, YouGov concur that there has been a narrowing of the gender gap - the swing to Yes among women since Tuesday's poll is a full 5%, while among men it's only 3%. But even though two polls are suggesting that the female Yes vote is closing the differential, that is still far from being absolute proof that it's really happening.
One thing that is really striking about a comparison of the two polls is that on the raw unweighted numbers, the Yes vote remains higher with Panelbase than with YouGov - but Panelbase's weighting procedures have harmed Yes, while YouGov's weighting procedures have harmed No. Of course to some extent that is for very good reasons - YouGov's raw sample significantly under-represents lower income voters, who are more likely to be in the Yes column. But if I was going to raise a question mark, it would once again be over Panelbase's recent introduction of weighting by recalled European election vote, which no other firm is doing and which significantly harms Yes - the 278 people who recall voting SNP in May have been downweighted to count as just 221 people.
By the same token, eyebrows might be raised at the fact that YouGov's weighting by Holyrood vote from 2011 has helped Yes - the 331 SNP voters in the raw sample have been upweighted to count as 375 people, while Labour voters have been significantly downweighted. But the crucial difference is that YouGov know for a fact that they have an in-built problem in their panel with having too many Labour voters from 2011, because they collected detailed information at the time, and a large chunk of their current panel were already with them back then.
Panelbase and YouGov now have something important in common - they're the only two pollsters so far who have taken the sensible step of introducing weighting by country of birth, which we know is a strong predictor of referendum vote. Both pollsters are now showing a clear lead for Yes among Scottish-born respondents, although as you'd expect the lead is slightly bigger with YouGov (54% to 46%, compared to Panelbase's 52% to 48%). And of course in both cases, those respondents have had to be upweighted while English-born respondents have had to be downweighted. If all six pollsters were doing this, it's reasonable to imagine that the average No lead on the Poll of Polls would be a touch lower.
Paradoxically, though, what's really setting the YouGov poll apart from Panelbase is the findings among respondents who weren't born in Scotland - YouGov say that 32% of people from other parts of the UK, and 46% of people from outside the UK, are now planning to vote Yes. Panelbase don't have exactly equivalent figures, but they're saying that only 26% of English-born respondents and 24% of respondents born outside both Scotland and England are Yes voters. If those figures are underestimates (and for what little it's worth, my gut feeling is that they are), that could partly explain why Panelbase are failing to detect the swing reported by YouGov.
Indeed, what's really encouraging about the YouGov poll is that, almost right across the board, demographic groups that have hitherto resisted the Yes message are swinging in our direction. It's not just women and people born outside Scotland - the Yes vote among over-60s has increased from 31% in Tuesday's poll to 38% today.
Irritatingly, YouGov have departed from their previous practice of showing turnout-filtered voting intention figures in their datasets. However, the likelihood to vote looks almost identical among Yes voters and No voters, so it seems reasonable to suppose that Yes have exactly the same 51% to 49% lead among definite voters (which would mean that the swing to Yes since Tuesday's poll is 5% among definite voters, 1% higher than among the whole sample). By contrast, Panelbase's turnout filter is helping Yes - without it the No lead with DKs excluded would be 2% higher.
Final thought : fresh from his red herring yesterday about "expectation management", Kenny Farquharson is now trying to set another hare running about how Yes supposedly need more than 51% in the polls to have a chance of winning, because of the large number of postal votes that were cast when No were still clearly in the lead. Not to put too fine a point on it, that's utter garbage. If you've already voted, and a pollster asks you how you intend to vote, you're highly likely to tell them how you actually voted. That means that it's a touch harder for either side to achieve a swing in the polls, but that any Yes lead still means exactly what it says.
When I pointed that out, Kenny said : "That assumes people with postal votes don't change their mind." Well, no it doesn't actually, but even if we assume for the sake of argument that some people are telling pollsters they plan to vote Yes when they have in fact already voted No (!), that is nowhere near the disadvantage for Yes that Kenny seems to think it is. Nobody voted by post until after the second debate, and the first two post-debate polls had Yes at 47%. If the postal votes reflect that position, and we assume that they account for roughly 1 in 5 of all votes cast, then Yes would only need to be on 51% of the vote by polling day to win. (Do the sums yourself if you don't believe me!)
YouGov :
Yes 51.2% (+4.4)
No 48.8% (-4.4)
That's better than I originally expected, and the reason is that the YouGov poll was actually slightly misreported in many media outlets - the split with Don't Knows included is Yes 47% (+5), No 45% (-3), rather than Yes 47%, No 46%. As a result, I've had to make a slight correction to the latest Poll of Polls update - the average No lead is now a further 0.2% smaller.
The less good news is that Yes only barely made it to 48% in the Panelbase figures, thanks to the effect of rounding -
Panelbase :
Yes 47.5% (-0.1)
No 52.5% (+0.1)
So that further deepens the mystery of why the Panelbase numbers don't appear to be budging at all (albeit from an all-time high position for Yes), at exactly the same time as the No vote appears to be in freefall with the formerly No-friendly firm YouGov. The gender gap has been more or less wiped out in the Panelbase poll - No lead by 53.2% to 46.8% among women and by 51.7% to 48.3% among men, which reverses a modest Yes lead among men in the last poll. It would be tempting to believe that the swing to Yes among women must be real and that the swing to No among men is an illusion, but there's a danger of wishful thinking in that kind of speculation - it's just as possible that normal sampling variation is at play, and that the Yes figure is a bit too high among women and a bit too low among men.
Nevertheless, it's true that if you isolate out the figures for women only, today's two polls are actually identical, because YouGov are also showing figures of Yes 47%, No 53% for female respondents. Moreover, YouGov concur that there has been a narrowing of the gender gap - the swing to Yes among women since Tuesday's poll is a full 5%, while among men it's only 3%. But even though two polls are suggesting that the female Yes vote is closing the differential, that is still far from being absolute proof that it's really happening.
One thing that is really striking about a comparison of the two polls is that on the raw unweighted numbers, the Yes vote remains higher with Panelbase than with YouGov - but Panelbase's weighting procedures have harmed Yes, while YouGov's weighting procedures have harmed No. Of course to some extent that is for very good reasons - YouGov's raw sample significantly under-represents lower income voters, who are more likely to be in the Yes column. But if I was going to raise a question mark, it would once again be over Panelbase's recent introduction of weighting by recalled European election vote, which no other firm is doing and which significantly harms Yes - the 278 people who recall voting SNP in May have been downweighted to count as just 221 people.
By the same token, eyebrows might be raised at the fact that YouGov's weighting by Holyrood vote from 2011 has helped Yes - the 331 SNP voters in the raw sample have been upweighted to count as 375 people, while Labour voters have been significantly downweighted. But the crucial difference is that YouGov know for a fact that they have an in-built problem in their panel with having too many Labour voters from 2011, because they collected detailed information at the time, and a large chunk of their current panel were already with them back then.
Panelbase and YouGov now have something important in common - they're the only two pollsters so far who have taken the sensible step of introducing weighting by country of birth, which we know is a strong predictor of referendum vote. Both pollsters are now showing a clear lead for Yes among Scottish-born respondents, although as you'd expect the lead is slightly bigger with YouGov (54% to 46%, compared to Panelbase's 52% to 48%). And of course in both cases, those respondents have had to be upweighted while English-born respondents have had to be downweighted. If all six pollsters were doing this, it's reasonable to imagine that the average No lead on the Poll of Polls would be a touch lower.
Paradoxically, though, what's really setting the YouGov poll apart from Panelbase is the findings among respondents who weren't born in Scotland - YouGov say that 32% of people from other parts of the UK, and 46% of people from outside the UK, are now planning to vote Yes. Panelbase don't have exactly equivalent figures, but they're saying that only 26% of English-born respondents and 24% of respondents born outside both Scotland and England are Yes voters. If those figures are underestimates (and for what little it's worth, my gut feeling is that they are), that could partly explain why Panelbase are failing to detect the swing reported by YouGov.
Indeed, what's really encouraging about the YouGov poll is that, almost right across the board, demographic groups that have hitherto resisted the Yes message are swinging in our direction. It's not just women and people born outside Scotland - the Yes vote among over-60s has increased from 31% in Tuesday's poll to 38% today.
Irritatingly, YouGov have departed from their previous practice of showing turnout-filtered voting intention figures in their datasets. However, the likelihood to vote looks almost identical among Yes voters and No voters, so it seems reasonable to suppose that Yes have exactly the same 51% to 49% lead among definite voters (which would mean that the swing to Yes since Tuesday's poll is 5% among definite voters, 1% higher than among the whole sample). By contrast, Panelbase's turnout filter is helping Yes - without it the No lead with DKs excluded would be 2% higher.
Final thought : fresh from his red herring yesterday about "expectation management", Kenny Farquharson is now trying to set another hare running about how Yes supposedly need more than 51% in the polls to have a chance of winning, because of the large number of postal votes that were cast when No were still clearly in the lead. Not to put too fine a point on it, that's utter garbage. If you've already voted, and a pollster asks you how you intend to vote, you're highly likely to tell them how you actually voted. That means that it's a touch harder for either side to achieve a swing in the polls, but that any Yes lead still means exactly what it says.
When I pointed that out, Kenny said : "That assumes people with postal votes don't change their mind." Well, no it doesn't actually, but even if we assume for the sake of argument that some people are telling pollsters they plan to vote Yes when they have in fact already voted No (!), that is nowhere near the disadvantage for Yes that Kenny seems to think it is. Nobody voted by post until after the second debate, and the first two post-debate polls had Yes at 47%. If the postal votes reflect that position, and we assume that they account for roughly 1 in 5 of all votes cast, then Yes would only need to be on 51% of the vote by polling day to win. (Do the sums yourself if you don't believe me!)
Subscribe to:
Posts (Atom)