Friday, July 18, 2014

TNS-BMRB poll : 'Undecided leaners' are breaking for Yes

As I pointed out a few days ago that the rounding in the ICM poll had flattered No slightly, I should in fairness also point out that the reverse has happened in the new TNS-BMRB poll.  For the whole sample, Yes have been rounded up from a position of Yes 43.6%, No 56.4%.  Once again, though, a significant difference is made if you simply remove the 3% of the sample who say they will definitely not be casting a vote in September, which strikes me as an eminently sensible thing to do - that in itself is sufficient to take us up to Yes 44.3%, No 55.7%.  

Among the 74% of the sample who say they are certain to vote, Yes have also been given a little boost in the published figures by the rounding - on the unrounded numbers it's Yes 44.6%, No 55.4%.  However, for the second TNS poll in a row, the highest Yes percentage of all is found among the 85% of the sample who say they are either certain or very likely to vote - on that measure it's exactly Yes 45.0%, No 55.0%.  That may be significant, because if I was going to hazard a wild guess as to what the turnout will be, I think I'd plump for something closer to 85% than 74%.

I couldn't resist a little peek at Political Betting last night, to see what inventive forms of denial the dark hordes were coming up with to explain away the apparent swing to Yes.  I've identified a couple of particular favourites.  First up is one from a certain Aberdeenshire Conservative activist...

"Deep breath, and relax.  Scotland is on holiday."

This presumably implies that it's only affluent No-voting Scots who are on holiday, and that TNS are utterly incapable of coping with that problem by means of their weighting scheme.  It's worth pointing out, though, that exactly the same Aberdeenshire Conservative activist once dismissed the results of a bad poll for the Tories on the grounds that "it's Christmas", even though it was in fact late November.  By the same token, I'm guessing that any bad poll conducted during any part of spring is an irrelevance because "it's Easter", and as for January or February - forget it.  Lovers are far too busy gazing into each other's eyes in anticipation of Valentine's Day.

And this was the other one -

"The Don't Knows are also Nos."

Hmmm.  That would have been a much better line if this poll hadn't specifically asked Don't Knows which way they are leaning, and found that they are breaking slightly more for Yes.  Adding in undecided leaners to the voting intention numbers has a small but significant effect across the board, although on some measures it would be disguised by rounding.  All of the figures below exclude the hard-core of undecideds who are unable to give an answer even when pressed.

Should Scotland be an independent country? 

Whole sample, includes undecided leaners :

Yes 44.3%
No 55.7%

Whole sample other than the 3% who will definitely not vote, includes undecided leaners :

Yes 44.8%
No 55.2%

Respondents who are certain or very likely to vote (equivalent to 85% turnout), includes undecided leaners :

Yes 45.3%
No 54.7%

Respondents who are certain to vote (equivalent to 74% turnout), includes undecided leaners :

Yes 45.3%
No 54.7%

One question mark that John Curtice has raised about this poll is that Yes have been weighted up from their position in the raw data by slightly more than usual.  That seems to be largely because TNS have had to make a bigger adjustment for past vote recall, due to there being more people in the sample who say they voted Labour in 2011 than who say they voted SNP.  In order to conclude that the weighting procedure is misguided, though, you'd have to believe that there is a significant problem of people being unable to accurately recall how they voted in 2011, which would flatly contradict evidence from other pollsters that recall of 2011 vote is much more accurate than recall of 2010 vote.  In any case, even if we assume for the sake of argument that people who recall voting Labour are being downweighted too much, that isn't the whole story by any means - TNS also dramatically upweight people who say they didn't vote in 2011, or who can't remember how they voted.  Both of those groups are more No-friendly that the overall sample.  So it's not immediately clear that the position would be that much worse for Yes if weighting by past vote recall was eliminated altogether.  SNP voters haven't actually been upweighted much at all - only from 224 to 228, which has a negligible effect on the results.  As usual, a big part of the reason for the disparity between the raw data and the published results is that the heavily No-friendly oldest age group has had to be downweighted significantly to bring them into line with their real population share.


  1. The polls will say that No are winning the day before the actual result shows 65 yes, 35 no.
    This isn't about truth, it's about sleepers doing what they are told.

  2. Christina D is a total fruitloop. Everything is bad for Salmond and Cameron is a hero heading for a majority of 200 becasue all her friends are OO, BNP, Brittania voters.

  3. leave them, a bad poll for us keeps the movement hungry ! The only Result that counts is the one from the 18th

  4. This wee bit of Scotland is on holiday in the Costas. Didn't stop me filling in an online YouGov poll today on the iPad, by the pool. The usual made-up fibs. Poor suckers think I'm in love with Ed the Red. As Mick Pork would say: *chortle*

  5. Juteman - if that's true, doesn't your pointing it out undermine the purpose of these "sleepers"?

  6. "The only Result that counts is the one from the 18th"

    I just despair at comments like that, and yet people seem to say it every twenty seconds on Twitter. It makes us sound like Michael Foot, rather than a campaign that two pollsters are showing on the brink of victory.

    I don't agree with Juteman either. It's perfectly possible that the polls will be out by a few per cent (and the unusual thing about this campaign is that we already know for a fact that some pollsters have their methodology wrong), but there's no way on Earth that Yes are going to win by a 2-1 margin if the polls are still showing them behind on the eve of the referendum. That sort of thing just doesn't happen in western democracies.

  7. " That sort of thing just doesn't happen in western democracies."

    The British State isn't threatened like this every day, James. With the amount of paedophiles and criminals in positions of power, they can be 'persuaded' to do what is required, or be exposed.

  8. I know the circumstances are very different but reading this reminds me of last year's poll to abolish the Seanad in Ireland. A month before the vote when the undecided voters were excluded the margin was 64% in favour of abolition and 36% against. There was a large number of undecideds though (32%). The actual result was 51.7% against on a turn out of 39%.

    We can't and shouldn't ignore polling evidence, but there are big unknowns in these referenda, and while we should keep our feet on the ground a big upset on the day wouldn't surprise me at all.

  9. I'm of the same opinion as James. If the last few polls ahead of voting day still show No ahead without any having Yes ahead, then a narrow No is most probable, although a narrow Yes possible too.

    If we go in with averages even, then Yes should take it by a clear, but possibly not big margin.

    If we go in with Yes on average ahead, then a clear Yes with a fairly big lead is the likely outcome.

    If we go in with Yes increasingly out in front / a fair lead, then a big majority for Yes it should likely be.

    Keep in mind that only in the last 2 weeks ahead of 2011 was it clear the SNP were going to win by a safe enough margin. However, the scale of the win was still underestimated.


    As an aside, even though Yougov are pro-Labour in Scotland due to methodology, with SNP heavily weighted down (as is the case for all UKGE VI polls using 2010 weighting), Ed Miliband continues to score lower than Dave C north of the border. For example today's for the Sunday Times:

    32% doing well
    64% badly

    21% doing well
    72% badly

    This is with the SNP getting a lower level in the subsets than average (36% Lab / 26% SNP).

    Ignoring exact levels, in the subsets, SNP has gained by 6 points in terms of it's battle with Labour since the beginning of the year, as part of a decline which followed a brief resurgence which peaked in early 2013. This was also seen in Holyrood VI. It is present in all other UK polling subsets.

    These are encouraging signs as they suggest Scots don't believe in Labour any more, even for Westminster. If that's the case, then a Yes to independence is much more probable.


    And one final thing...

    Polls of course always ask about what people think and iScotland's economy would be like.

    They of course don't ask what it would be like if it remained in the UK directly.

    State of Britain's economy
    17% Good
    50% Bad

    How personal finances will do in the next 12 months:

    16% get better
    34% get worse

    These are fairly standard figures. Independence may make people nervous, but they do not believe the union is offering them a bright economic future.

    Subsets yes, but pro-union party subsets, consistent, and quite stark in the message.

  10. there's no way on Earth that Yes are going to win by a 2-1 margin if the polls are still showing them behind on the eve of the referendum

    James, I take issue with your position. The notion of margin of error underwrites much of the discussion about polls, but these margin of error figures are for random samples. Unfortunately, most polls (telephone polls excepted) use quota sampling methods and it is nothing more than a polite fiction that such polls are in any way random.

    Quota sampling relies on the idea that if we know the characteristics of the wider population that we wish to poll (and therefore the characteristics of the a truly random sample) we can simulate randomness by selecting a polling sample that matches what we believe to be the characteristics of a random sample. Quota sample polls are therefore are not random, but more accurately are surrogates for random samples.

    So far so good, but there is a problem. Quota sampling depends on us accurately understanding the character of the wider population being polled – in mathematical terms we must understand the shape of the curve that describes the population being polled that we might select a sample population that corresponds to the shape of that curve. In brutal terms, quota sample polls are nothing more than an exercise in ‘curve fitting’.

    Now, as regards verifiable empirical data (e.g. age, gender, and wealth etc. fro the census) the characteristics and shape of the curve is largely uncontroversial. Although ICM’s inclusion of too high a proportion of English born respondents shows just how easy it is for pollsters to mis-shape even this most transparent part of the curve. However, that aside, the real problems come when pollsters try to understand the political character of the curve describing the population being polled.

    Until the mid-to-late 80s this had not been a problem as there were strong correlations between social and economic circumstances and voting intention. However, by the early 90s those relationships had broken down and pollsters were forced to develop new methods - in particular the use of recalled vote to establish a polling sample with the sort of political balance that a random sample would produce.

    To some extent this technique is a success, but resting as it does on the foundation stone of historic party loyalty/allegiance it relies on the assumption that the wider population share the sample population’s party loyalty/allegiance commitments – which is open to question where the sample populations are made up predominantly of the politically engaged.

    Remember, for the wider electorate the referendum is not necessary a party political matter. Witness the rise of Labour for Indy and/or the failure of the No campaign to secure across the board support from the Trade Unions. The point is: if party loyalty/allegiance is not an accurate determinate of referendum voting intention then the shape of the curve that the pollsters use may be significantly - and perhaps substantially - wrong and the resulting polls may very well be out by a factor of 2:1. I have to stress that I am not advancing the argument that the polls are so substantially wrong - I remain undecided on that matter - merely that they may be.

    Finally, as regards quasi-random telephone polls (e.g. Ipsos Mori), one has to question the use of telephone polls at all. Just to take my local area, the 2014 Lomond and Argyll telephone directory contains circa 20,000 residential telephone numbers, but the 2000 edition contained over 100,000. There is no doubt that it is possible to assemble a balanced poll sample from residential telephone numbers, but like a poll sample assembled from vegans the resulting sample is likely skewed by a common characteristic not shared by the wider population as a whole (the possession of a landline telephone in one hand and veganism on the other). The important point is that the possession of a residential landline is indicative of long-term domestic stability, which is a predominant feature of older No voters.

    1. There is another problem with this approach. If you now favour independence, psychology tells us it is more likely that you will remember voting in accordance with that previously. This means that your past vote recollection is coloured by present sentiment. As a surge in independence support will equal a "unrepresentatively" high SNP vote, it will be down weighted.

  11. there's no way on Earth that Yes are going to win by a 2-1 margin if the polls are still showing them behind on the eve of the referendum. That sort of thing just doesn't happen in western democracies.

    Just as a matter of accuracy, I'd take issue with that observation on principle. Something very like that happened in Glenrothes, at the by-election. Some think this was due to electoral fraud on the part of the Labour party, while others attribute it to a very late swing, but there's no doubt the final result was substantially at variance with the final opinion polls.

  12. Were there polls in Glenrothes? Genuine question, I can't clearly remember. I do recall there being polls in Glasgow East showing Labour ahead, but with Glenrothes all I can remember is a vague sense that everyone thought the SNP were slightly ahead (although Marcia later said that she could sense during canvassing that it wasn't quite happening).

    Perhaps I should have said "that sort of thing just doesn't happen in western democracies, except at the constituency/district level, especially by-elections". The reason being that pollsters can't seem to quite master the art of polling at constituency level, and also that very late swings are much more likely in by-elections, due to the fact that everyone knows they're 'free hit' elections for protest votes.

  13. Comres "Wisdom of crowds"

    Ed Miliband is likely to be prime minister after the next election

    July 2014 (from May 2013)
    Don’t know

    There's been a solid rise in 'disagree' and fall in 'agree' over the past year.

    You can be almost positive a No vote will give Scotland the Westminster Tories again. Likely in majority as they have just 'saved the union'.

    In the Scotland subset, just 20% think Ed will be the next PM with 42% disagreeing. SNP ahead again here too in VI.

    So Scots don't rate Ed and are not thinking Labour will win the next GE. Lots of evidence for it. This is just another example.

  14. James, yes there were polls in Glenrothes, showing the SNP ahead. Even exit polls I believe.

    I take your point that it's not a comparable situation though. I hope for Yes to pull ahead in the 2 weeks before the referendum. Otherwise I will be a nervous wreck.

  15. There were labour tellers admitting defeat in Glenrothes before the postal votes arrived. Blatant fraud by labour to save The Moron. And further proof that the Electoral Commission is a tol of the establishment.

  16. I really don't know what's going on. Yes is rampant, out and about, and No virtually invisible.

    Even here, a really, really Tory village, I passed 3 parked Yes cars I don't recognise (in addition to 5 or so belonging to Yes activists). No No car stickers. Only 3 or 4 Yes windaes, but again no No ones unless you count one with a wee UKOK car sticker on a kitchen window facing the back garden (I was delivering papers, so sue me).

    Whenever I go out I see more Yes windows and Yes car stickers. Overall I only ever saw one No car sticker over in Midlothian a couple of weeks ago.

    Yestival is packing them out. Rory the Tory got about 80 to his big gig this afternoon and most of them don't have a vote. The much-trailed "Flowers of the Union" song seems to have sunk without trace.

    Of course all these expressions of Yesishness only amount to about 0.1% of the population. Maybe it's just this shy No factor people are on about. But this does not feel to me like a country that is psyching itself up to vote No.

  17. And I forgot the bit about us delivering by activist hand to every bloody isolated farmhouse in the county, multiple times, while they're having to use elected representatives to canvass, are bussing up paid work experience kids from England to pose for the cameras, and paying commercial leaflet delivery firms to do the actual deliveries.

    How does this translate to a No win? It's against all reason.

  18. Another thing that will throw the recalled voting numbers out of sink, is that it surely doesn't take tactical voting into consideration.

    If I am a dedicated Tory, but decide to vote Labour to keep the SNP out, will I admit this to pollsters? I doubt it.

    Especially if I'm SNP who voted Lib Dem tactically, I mean, who would admit that now that their name is dirt in Scotland?

    Since we haven't had an independence referendum before, no one knows how to properly weight the demographics, throw the fact that we have votes for 16s and over.

    We also have the concerted effort to engage with areas that previously did not hardly engage with the political life of Scotland/UK.

    It also looks like it will be somewhere around an 80% turnout, again something that is unheard of in Scotland.

    In my mind most of the well off do vote and will tend to vote for the Tory, or perhaps (but no longer) the Lib Dems.

    So who are these extra voters and are they likely to lean more to Yes?

    I am convinced they are.

    The question then must be: If they are new to the political world are they likely to be signed up on (political) polling companies books ?

    Obviously not.

    So it may not be some conspiracy that is causing the disconnect between what we are all feeling or seeing 'on the ground' and what the polling companies are telling us.

    To sum up, no matter how good the polling companies are at trying to reflect Scotland's demographic, they are unable to legislate for the fact that a lot of people are becoming engaged in politics for the first time through age or through being encouraged that this vote counts.

    I think the reality is that Yes is already ahead, but this is not reflected in the Polls.

    If they were behind in all polls on the day before the vote?

    I might not be so confident, but I wouldn't lose all hope, however I wouldn't expect us to win 65% of the share.

  19. I would LIKE us to win 63-65% of the vote, because I believe that is about the level of support for independence if the "fear factor" was removed.

    We need some sort of bandwagon effect in the last two or three weeks for that to happen though. I'll settle for 51% so long as the unionists don't try to have the result set aside.