It's been a bruising few days for Blair McDougall and the other Abominable No-Men, as their hopes of a post-debate polling bounce have receded. We now have a second poll that was partly conducted after the debate, and the pattern is uncannily similar to the first one - the No lead has actually fallen among definite voters. And in this case, it's fallen to its lowest level of the campaign so far in any poll from the traditionally No-friendly firm TNS-BMRB.
Should Scotland be an independent country? (Definite voters only)
Yes 38% (+1)
No 46% (n/c)
Although that's only a 1% increase in the Yes vote share, it's worth remembering that the 37% vote recorded in last month's poll was 2% higher than it had been in any previous TNS poll - or to put it another way, tonight's poll has the Yes vote 3% higher than in any TNS poll prior to July. A rough calculation suggests that with Don't Knows excluded the position is likely to be Yes 45% (n/c), No 55% (n/c), although there's an outside chance that it might be Yes 46% (+1), No 54% (-1), which would also be a new record. We'll find out when the datasets are published, presumably tomorrow.
On the headline figures that cover the whole sample regardless of likelihood to vote (they even include people who say they are absolutely certain not to vote!) there has been a predictable reversion to the mean since the last poll which saw the No lead slump to a record low by a huge margin. The bad news for the No campaign is that while tonight's figures may not be the worst for them in a TNS poll, they're the second worst. Last month's poll was the first time in the campaign that Yes had been higher than 41% in a TNS poll after Don't Knows were stripped out, and tonight is the second. That makes it very difficult to argue that Yes haven't made further progress since the spring, unless there have been some highly coincidental margin of error effects in both of the last two polls.
Should Scotland be an independent country? (Whole sample with Don't Knows excluded)
Yes 42% (-2)
No 58% (+2)
With Don't Knows taken into account, Yes remain on 32%, which is their record high for the campaign - prior to last month's poll they had never been higher than 30% (or at least not since TNS introduced an enormous methodological change).
When the datasets are released, the first calculation I'll be making is voting intentions with the small number of definite non-voters stripped out - in both of the last two TNS polls that in itself was sufficient to increase the Yes vote by the best part of 1%. And that gives a clue as to why the figures for definite voters only (which TNS are increasingly giving parity of esteem to) may be the more meaningful ones - they equate to a turnout in the low 70s, which is perfectly plausible. If you think that the turnout is more likely to be around the 80% mark (or a bit higher), then the relevant figures will be for those who say they are very likely or certain to vote, and we won't find out what those are until tomorrow - although it's reasonable to assume that they'll be more favourable for Yes than the numbers from the whole sample.
As I always point out, good results for Yes in polls from TNS and Ipsos-Mori are worth more than good results from other pollsters, because those two are the only firms that don't rely on volunteer online panels for their referendum polling, and that actually seek out a sample in the 'real world'. It's hugely encouraging that both are now showing Yes at a record high for the campaign among definite voters (42.5% in the case of Ipsos-Mori, 45% or 46% in the case of TNS) as we enter the crucial last few weeks.
And what does the TNS poll tell us about the post-debate polling landscape more generally? It slightly increases the likelihood that the Survation poll showing an unusually high No lead was an outlier caused by random sampling variation - but the emphasis is on the word 'slightly', because what we really need to be sure is another poll that was entirely conducted after the debate, rather than only partly.
Final thought - as far as we know, TNS haven't yet joined the new orthodoxy of weighting by country of birth. That may not be such a huge issue for them, because a face-to-face pollster ought to find it easier than an online pollster to put together a representative sample - but it would still be a very wise idea to weight by country of birth to be on the safe side, because it's such a strong predictor of referendum vote.
* * *
SCOT GOES POP POLL OF POLLS
To state the bleedin' obvious, the only reason this update of the Poll of Polls shows a slight increase in the No lead is that TNS are represented in the sample by their headline figures - if they were represented by the numbers for definite voters (as is the case for Ipsos-Mori), the No lead would have fallen back again.
MEAN AVERAGE (excluding Don't Knows) :
Yes 42.4% (-0.4)
No 57.6% (+0.4)
MEAN AVERAGE (not excluding Don't Knows) :
Yes 36.5% (n/c)
No 49.5% (+0.7)
MEDIAN AVERAGE (excluding Don't Knows) :
Yes 42.5% (-0.3)
No 57.5% (+0.3)
(The Poll of Polls is based on a rolling average of the most recent poll from each of the pollsters that have been active in the referendum campaign since September 2013, and that adhere to British Polling Council rules. At present, there are six - YouGov, TNS-BMRB, Survation, Panelbase, Ipsos-Mori and ICM. Whenever a new poll is published, it replaces the last poll from the same company in the sample. Changes in the Poll of Polls are generally glacial in nature due to the fact that only a small portion of the sample is updated each time.)
+1 is soaring? I swear you do this to troll ;)
ReplyDeleteConfirmed BT claiming their lead increasing?
ReplyDeleteOn yerself James!
ReplyDeleteDaily record page back online:
ReplyDeletehttp://www.dailyrecord.co.uk/news/scottish-news/support-no-vote-stalling-says-4039764#.U-qgCO_RyJM.twitter
Blinking: I think that tweet is based on a comparison of the full panels of both polls, so a fair comparison would think. What would be wrong would be to compare the full sample to the certain to vote sample, as you suggest.
ReplyDelete"How come Better Together are claiming the no vote has increased by 4%?"
ReplyDeleteBecause that's what the overall figures actually say. What James has quoted here is only for definite voters.
Now of course the definite voter figures are interesting, but I think it's pretty reasonable to suggest we should at least be acknowledging what the overall figures say as well. Maybe that's coming in an updated version of this post as he does say there are more details to follow.
I think James's "outside chance" is a very outside one.
ReplyDeleteMy calculations are -
Excluding undecideds (changes from the TNS poll published on 14 July in brackets)
All respondents – Yes 42% (-2%) No 58% (+2%)
Certain to vote – Yes 45% (+1%) No 55% (-1%) – (rounded %s are the same, but underlying figures show the shift – actually 0.7%)
Certain to vote – 73% (-1%)
Undecideds
All respondents – 23% (-4%)
Certain to vote – 16% (-2%)
So a poll that both sides can spin their own way.
Within the campaign period certain to vote is usually used by pollsters so the 38/46 should really be the headline figure
ReplyDeleteTwo points
1. the Herald and Record headlines were pulled because they were pro-Yes so I now believe the polls are being manipulated and being used to destroy the confidence of the Yes campaign.
2. Better Together always get details first so are able to spin any poll as favourable to them.
We should assume the polls are no more reliable than any other BT propaganda but how to stop confidence draining away?
Could Yes commission a poll? Or perhaps we could produce a poll from RIC canvassing?
Wise words from oldnat but not really on message.
ReplyDelete"Wise words from oldnat but not really on message."
ReplyDeleteEvery Cabinet needs its Ken Clarke - as David Cameron will now discover.
I don't want an indy Scotland so that I can "stay on message"! I want it so that I can remain "off fucking message!"
ReplyDelete"And what does the TNS poll tell us about the post-debate polling landscape more generally?"
ReplyDeleteI'd be careful reading too much into that, given that the survey was "largely conducted during the games", prior to the debate.
As far as I can see it carried on for two days after the debate, which is non-trivial.
ReplyDeleteJames: I was going by the statement in the Daily Record article. Is there any further information on fieldwork dates? I'd agree 'some' is non-trivial, but it may not be significant, say ~10-20%.
ReplyDelete"The survey was largely conducted during the Commonwealth Games in Glasgow, with some interviewing taking place after the August 5 televised debate between First Minister Alex Salmond and Better Together leader Alistair Darling."
another excellent poll
ReplyDelete5 weeks to go and a very healthy lead
no commonwealth games bounce which was Alex's plan and we are still to see the effects of Alex's pandas and aliens driving on the right hand side of the road
all very very encouraging
Another anonymous out of touch twit who simply doesn't understand that labour were just as confident and just as complacent in 2011 when the polls utterly failed to reflect the reality on the ground
ReplyDeleteno post-debate bounce for Darling and the unionist media's demented shrieking about currency, and we are still to see the effects when Darling gets asked why he supported the Iraq War and scaremongered about Weapons of Mass Destruction.
all very encouraging.
Even more encouraging are the mass canvass results for Yes. Happily the BritNat twits who are too complacent and ignorant to care about GOTV and the ground campaign
The last poll in 2011 that had Labour in the lead was 27 march.
ReplyDelete"The last poll in 2011 that had Labour in the lead was 27 march."
ReplyDeleteYeah, but we don't need a result of Y+18%, +1 vote will be good enough. ;-)
This referendum has convinced me that polling is simply another propaganda tool.
ReplyDeleteIt's all gone Salmond -shaped for the Yesmen.
ReplyDeleteDavid Halliday - I believe so, which is why eg. MORI use "certain to vote" as their headline number.
ReplyDeleteThere are lots of links to actual datasets in the tables on this page:
http://en.wikipedia.org/wiki/Opinion_polling_in_the_Scottish_Parliament_election,_2011
Brian:
ReplyDeleteThanks but I think if you actually follow the links you'll see overwhelmingly most of them are dead.
Panelbase poll in the field according to my pal
ReplyDeleteMight get it for this weekend
So yes have lost then.
ReplyDeleteIf you're willing to take respondents' word about how definite they are to vote at face value, you also have to take their word about which way they're going to vote at face value.
This despite their vague anc shifting weightings, both before and after the poll.
ReplyDeleteI see yes vs No amongst men is almost back to 2011 levels when Yes went ahead (including in TNS). 48% Yes for men now.
ReplyDeleteLadies still being coy. This of course was the 2011 pattern. Men 'moved' first to close the gap then women shifted to hint at the landslide coming. Of course the shift actually occurred 2007-9 then hid for 2 years meaning it was more men became honest (again) first, then women.
Yes has 84% Certain to vote, but No only 74%. So that's 36% saying they are certain to vote and planning No for now. That's quite a difference - why the reluctance among No to actually vote No?
Unfortunately TNS haven't asked the question 'Will you yet change your mind'. That of course gives <30% solid No as per ICM's 'completely against independence' numbers and agrees with No in canvassing.
No better hope that women, for the first time in the political history of Scotland, don't end up largely voting the same way as men. We know that women are less firmly against indy than men (ICM 1-10) and distrust Westminster as much or more. They are also saying they are more Scottish / less British (SSAS). I wonder what they'll do. If they do the same as 2011, then Yes will in all probability take it. It's pretty much that simple.
48% ex DK for men I mean.
ReplyDeleteYouGov IndyRef poll in the field this morning. Very many questions about Salmond after a NO vote
ReplyDeleteI note also that for the first time in TNS, Yes is higher in the unweighed base than it is in the weighted base. The opposite for No.
ReplyDeleteJames - I believe you've noted this occurring in other polls? I wonder what's causing it if it is a new phenomena.
We can see that TNS struggle much more to get people under 54 to talk to them; the group most supportive of Yes. I say struggle to get them to talk as TNS is not self selecting; people need to actively agree to take part once they open their door and are asked if they want to.
The data suggest those age groups most likely to vote Yes are the age groups most likely not to partake in the poll - men in particular.
Well, this is fascinating.
ReplyDeleteLarge groups of people are not answering TNS polls it seems and extent of this varies with time in response to events. The groups not answering are those most supportive of independence and the SNP; younger generations.
How can you tell? Well, simply from the total up-weighing needed for each age group. The more a group needs up-weighting relatively, the more they must not be responding.
If there was a continuous problem reaching a particular group you'd not see any trend with time as the up-weighting needed would always be fairly consistent or at least bounding up and down a bit with no obvious trend with time.
In 2009 when the SNP were hitting >40%, very little up-weighting was needed. No referendum on the cards and the heat on Scotland wasn't too bad.
In 2010 election approached time (heat turned on) when Labour were apparently ahead, the up-weighting needed hits a huge spike; large numbers of people were refusing to give TNS their views - presumably SNP voters.
This then plunges as we head into 2011 and the polls start to show the SNP gaining. It's not maybe simply lying (this did happen), but a large component lying by not answering at all. When these people started talking, they were SNP...
In 2011, the need for weighting dropped massively (but not to 2009 levels) and of course Yes went ahead as a result; more Yes people were answering.
Since then, the up-weighting has climbed back and is reaching record highs, which strongly suggests more and more people are shutting the door on TNS and these people are Yes. The fact that Yes is rising and No fell even with that is quite something.
Yup, the higher the total up-weighting needed to correct for those refusing to answer the poll, the lower the Yes vote.
ReplyDeleteYou can even see the 2012-13 No peaks - upweighting peaks again here then starts to fall gently in line with Yes recovering.
Yup, the higher the total up-weighting needed to correct for those refusing to answer the poll, the lower the Yes vote.
ReplyDeleteYou can even see the 2012-13 No peaks - upweighting peaks again here then starts to fall gently in line with Yes recovering.
Why do the trolls mix up what a poll company predicts with what is happening in reality? They are at best blindly trying to predict where the light switch is in a darkened stadium. At worst they are using their psephology to tamper with the public's perceptions.
ReplyDeleteJames Kelly highlights the direction of the trends in their figure and bases his views on l'actualite that the people are moving towards 'Yes'.
Thanks SS, I now understand the way polling works LOL
ReplyDeleteWhat an absolute crock of shit it all is.
'upweighting' people that vote, of course you are going to get the sorts of figures no get...as that is the only figures they can go on.
People pay money for this?
Thanks SS, I now understand the way polling works LOL
ReplyDeleteWe might also ask ourselves why MORI struggled to get men (more Yes) to answer the phone too, with this problem peaking when No peaked / yes toughed. As the number of men agreeing to be polled has climbed, so Yes has risen again and No has fallen.
The same age issues are also evident in addition to men; the more MORI have managed to get a representative sample (i.e. they've not been struggling with some groups, notable younger people and also men), the higher the Yes is.
This does strongly suggest pollsters do have a problem with getting Yes people to answer polls.
That might annoy us polling geeks, but in the end it's them voting Yes on the day that's important.
Anyway, thank the no-campaign for demonising Yes with all the 'anti-english xenphobic racist' stuff. You want to get people to say no to answering indy polls, that's how you do it.
I'm going to look further into it, but the TNS data set is extensive, covering many years and quite stunning. It is also non-self-selecting which is ideal for getting an idea of abstention.
People who choose for example not to sign up to online panels is likely to be much harder to detect. I'm going to have a look though.
"We might also ask ourselves why MORI struggled to get men (more Yes) to answer the phone too"
ReplyDeleteWe might indeed since I personally was contacted by MORI three over a period of many months that clearly covered multiple polls. Each time they were only interested in the 16-24 cohort.
The last time the Mori employee also promised to include me in a future poll after I had pointed out to them they had contacted me three times previously and they weren't interested.
Still waiting to hear from them curiously enough and that last contact was over a month ago.
"This does strongly suggest pollsters do have a problem with getting Yes people to answer polls."
And that at least some of those pollsters don't seem capable of conducting those polls particularly competently.
Here's what I dont understand about all this shy Yes stuff.
ReplyDeleteIt seems the whole site agrees, even No contributors agree, that Yes are more outgoing , more excited, more anxious to campaign, keener to put Yes in their window, to celebrate loudly on social media when a single NO sinner is converted, fo knock on doors and are generaly desperate to publicise their views and quite publicly find the whole campaign a tonic for the soul.
Moreover there is indeed hard polling evidence that Yes are happy with the length of the campaign and are enjoying the campaign while No voters are doing their best to ignore it and wish it had been over some time ago.
Yet we are aked to believevthat a large portion of these same keen-to-participate keen-to convert Yes voters who put all these Yes posters in their windows apparenty bury their head, hide their views and/or run a mile at any prospect of airing their views iIN CONFIDENCE to a pollster? Meanwhile the bored disillusioned No voters who want it all over are apparently much more ready to give time and energy to pollsters.
More likely surely that the bored frustrated No voters, who are not enjoying the campaign one bit, would be the ones to bypass spending time airing ther views.
It doesnt stack up.
Yet we are aked to believevthat a large portion of these same keen-to-participate keen-to convert Yes voters who put all these Yes posters in their windows apparenty bury their head, hide their views and/or run a mile at any prospect of airing their views iIN CONFIDENCE to a pollster?
ReplyDeleteOh yes it does stack up - because these are NOT the same voters. The voters flying the flags are not at all shy, but there are many more keeping their heads down.
I'm a man. When i'm watching the TV and the phone rings, i pick it up.
ReplyDeleteIf it is computer generated call, or someone on the other end says ' Hi, i'm.....', i put the phone down.
Why would anyone welcome market research phone calls? I categorise these as nuisance calls and got them blocked years ago.
ReplyDeleteExpat: I have two yes stickers on my car and one in my house window, I have twibbons on facebook and twitter. I post indy comments on both daily. I wear my badge every day.
ReplyDeleteMy sister who is every bit as yes as I am won't even click the like button on the Yes Scotland page on facebook in case some no voting friend or workmate sees it and unfriends her.
It's just how it is.
Maybe she's just more insecure than I am, haha.
It doesn't stack up.
ReplyDeleteMaybe it doesn't.
I'm just reporting what I see in data and analysis of data looking for correlations is my day job (albeit production chemistry rather than social sciences).
There does seem to be a strong correlation between being able to more easily achieve a sample without having to heavily up-weight certain groups and a higher Yes.
This pattern also seems to following polling trends in response to events.
I've speculated as to the cause.
Here's a list of things I've been called for saying openly to people I support indy:
Racist
Anti-English
Supportive of a dictator
Xenophobe
Vile
Abusive
Nasty
Stupid jock
Economically illiterate
Living in a fantasy braveheart world
etc.
And I've not been called these by a few trolls on the internet, but by prominent politicians and media figures. Live on TV, in the papers etc. Even in Parliament.
I'm made of quite stern stuff though and can argue my case. However, even then I've been shy Yes, particularly in the beginning before I felt I could fight anything thrown at me with reasoned argument. Nice Mrs McPhee from bathgate is likely less so.
It is a well known fact that people lie in polls, even if only to avoid embarrassment about minor things. People say they voted when they didn't, they say they plan to vote when chances are they won't. They say they backed the winner when they didn't. They say they voted Labour when they, out of self interest, voted for the Tories.
If you ask them if they ever drove under the influence, chances are they'll say no even if they did. On sensitive issues which they can be attacked for they are particularly reticent to the point they may decline to be asked at all. Breaking up Britain is about the most sensitive political issue you can get.
"Oh yes it does stack up - because these are NOT the same voters. The voters flying the flags are not at all shy, but there are many more keeping their heads down."
ReplyDeleteThat's true, of course, but the argument seems to rest on the idea that the social environment makes Yes voters more likely to keep their head below the parapet than stick their head up and risk a negative reaction. But if Yes has more vocal campaigners you would think it should be the opposite - that there are a greater number of people ready to criticise No voters which therefore makes the social environment less amenable to those expressing No sentiments.
Anecdotally if someone pointed a gun at my head and forced me to make a judgement call on it I'd say the reaction to No supporters seems to be harsher than for those on the Yes side.
That's true, of course, but the argument seems to rest on the idea that the social environment makes Yes voters more likely to keep their head below the parapet than stick their head up and risk a negative reaction.
ReplyDeleteNo it's not the social environment we're talking about here; many people in relaxed social environments are indicating a shift to YES. It's the reactions to authority that are pertinent, because that authority constantly vilifies the whole concept of independence. Anything that has an "official" status becomes suspect.
If the No supporters on here seriously don't understand the difference between political activists/grass roots of parties and run of the mill voters, (who self-evidently hugely outnumber activists/grass roots for any party) then you have to wonder if every election and referendum baffles them.
ReplyDeleteThey are not even close to being the same thing.
A grass roots campaign is also a very different beast indeed to the 24/7 angry shrieking from the westminster bubble media and some 37 daily/national papers in scotland who have made no attempt whatsoever to disguise their attacks on the Yes campaign and Yes supporters.
On of the biggest points about a grass roots campaign is that it grows below the surface (the clue is in the name) and completely bypasses the overt and judgemental media landscape. it does this far more subtly by using things like word of mouth and meetings up and down scotland that are being packed out. Those meetings are where undecideds can sit in peace listening to the arguments and know they will not be bothered by a rabid and shrieking media. That intense incessant hostility from the media will obviously make voters keep their cards close to their chest.
That was the choice of the No campaign who the decided on Project Fear and scaremongering. Let's not forget it was also the choice of SLAB when they had a negative and hostile campaign in 2011 AND 2007. (with some of the precise same idiots running both SLAB campaigns and now involved in Project Fear)
Turns out that relentless negativity does indeed succeed in creating Shy voters who are on the receiving end of it. Those results have already proven it.
Why on earth should that be a surprise to anyone since the pollsters STILL use corrections they had to introduce for Shy voters from as long ago as 1992.
another excellent poll for the no camp tomorrow
ReplyDeletesurvation\daily record
1000 women polled
34% yes
50% no
16% undecided
Good stuff
Is that not a very low no vote among women?
ReplyDeleteSLAB trotted out the same seemingly low polling scores for women voters in 2011.
ReplyDeleteThen there was the SNP landslide which showed just how much women were being affected by shy votes.
The complacency of the No campaign is wonderful to behold after they did the precise same thing twice before in 2007 and 2011.
No are merely pushing their already laughable grass-roots and GOTV organisation ever lower and No voters will have little motivation to go out and vote if they seriously believe it's already in the bag for them.
The Yes campaign well never reach that level of complacency. We learned the lessons from 2007 and 2011 and we've worked most of our lives to see this vote.
"Is that not a very low no vote among women?"
ReplyDeleteThe No lead has fallen by 3% among women since the last Survation poll, yes. So not such an "excellent poll for the No camp" - but you could probably have guessed that our resident No trolls wouldn't exactly be telling you the whole truth.
"Then there was the SNP landslide which showed just how much women were being affected by shy votes."
ReplyDeleteYouGov's final polls in 2011 got the constituency result almost spot on, so shy SNP voters can't have been much of an issue. The final rounded result was SNP on 45, Labour on 32. YouGov had:
21st of April: SNP 45 / Labour 32
28th of April: SNP 42 / Labour 34
4th of May: SNP 42 / Labour 35
So the 21st poll was pretty much as accurate as a poll gets and the other two were within or just outside the usual 3% margin of error. There isn't much room for "shy SNP voters" there.
Reporting a 7% SNP lead when the actual result was a 13% SNP lead is "almost spot on"? Let's get real here. And you can't just gloss over the mind-bogglingly huge error on the list vote.
ReplyDelete"Reporting a 7% SNP lead when the actual result was a 13% SNP lead is "almost spot on"? Let's get real here. And you can't just gloss over the mind-bogglingly huge error on the list vote."
ReplyDeleteA few points:
1. The margin for error refers to the percentage a party receives, not the gap between parties. If two parties have identical "real" support it's entirely possible for a legitimate poll to show them 6 points apart - yet you've just quoted an example of the gap being 6 points off as if that proves the polling was nonsense. That's not even considering the 95% confidence.
Whichever way you look at it there's no proof that there was some "shy SNP voter" effect in those numbers. It's all entirely consistent with an accurate methodology.
2. The constituency poll on the 21st is undeniably spot on (which is the point at which postal ballots had been sent out - i.e. 20% of the counted votes in the election).
3. The regional vote was off, but it's hard to imagine why this "shy SNP voter" effect would exist in the regional vote but not in the constituency vote. Are people only intimidated from indicating a preference for the SNP when they're asked about their second vote in a Scottish election? That would be one of the oddest polling phenomenons we've ever seen and if you could prove it I'd advise giving up this blog and drafting a journal article on the subject.
"If two parties have identical "real" support it's entirely possible for a legitimate poll to show them 6 points apart"
ReplyDeleteNot within a 95% confidence level it isn't - otherwise a 6% lead would be considered a statistical tie, which isn't the case. Admittedly it's a very common misconception.
"The constituency poll on the 21st is undeniably spot on"
Oh well, as long as YouGov are only wildly inaccurate in two polls out of three, who's to quibble?
Just to be clear...
ReplyDeleteThe 'problem getting respondents' proxy I used (for shy) showed a huge peak when Labour were 'apparently' ahead in polls in 2010 / early 2011. This then dropped sharply as 2011 approached which would suggest the respondent problem reduced markedly close to the election. That might help explain how suddenly the SNP appeared to gain, with shy vote hitting a minimum just before the election and so not having much of an impact in final polls. People started answering who did not before.
For the TNS data there does seem to be a close correlation between the apparent inability to reach respondents from certain (more Yes) groups and a higher No. The pattern goes up and down like the polls and correlates linearly when you plot one vs the other all the way back to 2007>.
A shy vote would affect all polls, although finding evidence for it in online poll data may be harder (I've not looked as yet, but see it in MORI telephone) as how do we know these people volunteered for panels in the first place? We would need them to have both done so and then stopped responding to requests in large numbers to the extent that remaining members of similar age/sex were insufficient to plug the gap and abnormalities started appearing.
Shy vote, if present, would always be more obvious in a non self-selecting poll where obtaining respondents required the most effort / time. That is TNS chapping doors. Even for MORI, the ability to just keep phoning means filling gaps is easier and quicker. Of course you are filling gaps with non-shy.
Note shy does not mean SNP alone. It could be shy 'don't vote normally' or shy Labour Yes. In the latter case, that would result in a lower Yes in your 2011 recalled vote Labour group.
Only time will tell. If true it means Yes may do better than polls conclude right now. How much better is not possible to predict.
The fact is when TNS (and MORI) don't have to up-weight, they get much higher Yes and that's all we know.
Note if there is a Yes, we can expect polls taken afterwards to show more people saying they voted Yes than did.
ReplyDeleteThat would be a normal response and is typical of 'historic' wins. Plenty of examples of 'I backed the winner honestly'; the 1997 GE being a classic.
People are never fully honest with pollsters, whether it be embarrassment, something is sensitive or simply because people are complex and not every issue is black and white so asking them different ways gets different answers.
Pollsters and polling geeks would love it if people of every view responded equally and honestly. They don't and what effect that has on final vote is very difficult to predict.
"Not within a 95% confidence level it isn't - otherwise a 6% lead would be considered a statistical tie, which isn't the case. Admittedly it's a very common misconception."
ReplyDeleteI'm really not sure what you're trying to argue here because on the face of it that comment doesn't make any sense. A 95% confidence interval simply means it will be within 3% of the actual level of support 95% of the time (i.e. in 19 out of 20 polls you conduct). The other one poll will be outside that range.
What we have in those polls are (with the admitted qualifier that we don't know the rounding) two polls that are within 3% of the actual total the SNP and Labour got in the election and one poll that's virtually identical to the actual total. You calling that "wildly inaccurate" or pointing at the 6 point gap as if it's proof the polling was methodologically flawed is simply incorrect. There is nothing in those results that necessitates a theory about "shy SNP voters" because they're entirely consistent with a broadly accurate polling methodology that's simply subject to the random variations you get in all polls.
"I'm really not sure what you're trying to argue here"
ReplyDeleteIf you don't understand the point I'm making, how do you feel able to tell me with such certainty that I'm incorrect? As it happens, the point I'm making is precisely the one you think I'm making - that a 6% error in the lead is outside the 95% confidence level. You think it isn't - as I said, that's a very common misconception.
There is no doubt whatever that YouGov had their methodology wrong in 2011. They must accept that themselves (privately at least) because they've since changed it radically.
By the way, you keep going on about "shy SNP voters" as if that's a point I've made - I haven't. I'm simply pointing out to you that you're wrong about YouGov's results being defensible.
"a 6% error in the lead is outside the 95% confidence level."
ReplyDeleteThe reason I don't understand the point you're trying to make is that your sentence above makes absolutely no sense. You seem to be suggesting that a 95% confidence interval means "95% of 3%" (and therefore that the margin of error is actually less than 3% so a gap of 6% is impermissible). I gave you the benefit of the doubt on the first comment - that you might have simply been trying to make a completely different point and it came out in the wrong way - but you've just repeated the same thing.
The 95% confidence interval doesn't refer to the size of the margin of error. What it means is that if you run 20 polls you'll have 19 of them within 3% of the actual total for each party, while the other poll will be outside of that range. In other words, if you run 20 polls one of them is bound to be unreliable through random circumstance - i.e. 95% of polls are reliable, 5% aren't; you have "confidence" in 95% of the polls conducted.
What it absolutely does not mean is that the 3% margin of error is "only 95% of 3%". Instead of being argumentative about it I'd suggest you take the point on board - you're running a blog about polling after all.