Drama as No-friendly TNS poll finds half of the Scottish public want an independence referendum

I've been having a look at the newly-released datasets from TNS to see if there is any potential explanation (other than the obvious one of data collection method) for why they are contradicting three other pollsters in showing a big swing to No.  What leaps out at me are the weightings for recalled Holyrood vote, and the highly unusual way TNS treat respondents who say they didn't cast a vote last year.  Most pollsters who weight by recalled vote do not try to upweight abstainers until they reflect the actual abstention rate.  There are two very good reasons for that approach : a) people are embarrassed to admit that they didn't vote, meaning a significant proportion will lie and claim they did turn out, and b) people who do openly admit they didn't vote are particularly unlikely to vote again in future elections anyway.

That being the case, you'll quickly spot the problem in the fact that only 27% of the unweighted TNS sample either said they didn't vote last year or can't remember how they voted, and that TNS decided to massively upweight that group to 42%.   It looks highly likely that disproportionate weight has been given to a hard-core of non-voters.  That doesn't explain all of the swing to No by any means - there is movement in that direction almost across the board among voters for most parties.  But the swing among abstainers from last year is very large - they've gone from being virtually split down the middle in the last TNS poll to being in favour of No this time by a 21-point margin.  The massive upweighting will obviously have artificially magnified the effect of that.

The biggest downweighting on the recalled vote is among people who say they voted SNP - 38% of the unweighted sample were SNP voters, and they were scaled down to count as just 27%.  That obviously has a significant detrimental effect on the reported Yes vote.  It's not unreasonable to speculate that 'embarrassed abstainers' who falsely claim to have voted last year may have defaulted to saying they voted for the winning party, so while it's possible that TNS may have interviewed too many SNP voters by chance, it's also possible that this group has been downweighted too much, leading to distorted headline numbers.

In addition, there's a very familiar problem with respondents who recall voting for an "other" party - meaning a party other than the SNP, Tories, Lib Dems or Labour.  This small group often ends up being very sharply downweighted, because people are asked how they voted on the constituency ballot, but instead find themselves recalling their vote for the Greens or UKIP on the list.  In the new poll, this had led to them being scaled down from 3.4% of the raw sample to count as just 0.6%.  It's blindingly obvious that TNS aren't giving them sufficient weight, and as it happens, they are the only group that didn't show any movement to No at all.  They also broke only very marginally for No overall.  If there had been a more realistic target figure for "others" to take account of the confusion between the constituency and list vote, this factor alone could conceivably have slightly reduced the reported swing to No.

You'll have seen a lot of hysterical coverage today about how this poll supposedly shows that the public don't want an independence referendum.  You probably won't faint with amazement to discover that it shows no such thing.  Excluding Don't Knows, 49% of respondents chose one of the four pro-referendum options provided by TNS, and 51% chose the sole anti-referendum option provided.  That's within the margin of error, so must be regarded as a 'statistical tie', and is strikingly similar to the findings of recent Panelbase polls which have also shown voters split down the middle.  It's also worth pointing out that if the TNS poll does turn out to be a rogue poll with too many No voters in the sample, the 49% in favour of holding a referendum is likely to be an underestimate.