A few more bits and bobs from the TNS-BMRB poll

One thing that always amuses me when I read through the TNS datasets is the crossbreak that shows how the people who are certain not to vote in the referendum are planning to vote in the referendum.  In this case, 50% of definite non-voters are "planning to vote No", 10% are "planning to vote Yes", and the remainder are "undecided".  What exactly is the point of the voting intention numbers taking account of people who have already decided not to vote?  I can only think of two realistic explanations - a) it's assumed they might not have properly understood the question about how likely they are to vote (in which case it would probably be better to improve the wording of the question), or b) it's assumed they might not be telling the truth.

For what it's worth, though, if we take those people at their word and strip them out of the voting intention numbers, it's enough to reduce the overall No lead by 1.2% at a stroke (the 58.6%-41.4% lead becomes a 58.0%-42.0% lead).

Although this shouldn't really matter, it's intuitively encouraging to see that the Yes vote in this poll is very nearly as high on the raw numbers as it is on the weighted results.  That doesn't often happen, because Yes support is concentrated in groups that typically need to be upweighted in polls (such as younger and lower-income people).  It's also a relief to see that respondents' vote recall on the raw numbers reflect the fact that the SNP won the 2011 election comfortably.  There have been so many previous TNS polls in which more people recalled voting for Labour than for the SNP that I was beginning to wonder if false memory might be mucking up the weighted results, but thankfully that's not a concern in this particular poll.

As we've discussed a number of times before, TNS have a very odd weighting scheme which essentially assumes a 100% turnout.  People who say they didn't vote in 2011 or who can't remember how they voted are sharply upweighted to match the rate of abstention in the 2011 election.  But for the life of me, I just cannot understand the logic of treating people who can't remember how they voted as if they were non-voters.  Again, the assumption must be that all of them are lying or misremembering, but even if that's the case (highly unlikely), how can TNS possibly calculate the correct target figures for the proportion of 2011 non-voters who can be assumed to now be wrongly saying that they voted but can't remember how?  It's hard not to conclude that there must be a hell of a lot of random guesswork going on here.  As it happens, people who say they can't remember how they voted in 2011 almost always break heavily for No in TNS polls, which means that if the upweighting doesn't have a rational basis, then it will be artificially boosting the No lead (albeit only by a modest amount) in the headline numbers.  In this case, the 92 'can't remembers' in the raw numbers are upweighted to count as 135 'virtual' respondents.

The reality is that a lot of people who didn't vote in 2011 are probably embarrassed to admit that to pollsters, which means that what TNS are doing is fundamentally misconceived - if they want to ensure non-voters are properly represented, they should really be looking for some of them in the "voted Labour/SNP/Tory" columns.  The mistake may not be having a huge net impact on the headline numbers, because people who openly admit to not having voted in 2011 (as opposed to the 'can't remembers') actually split between Yes and No in much the same way as the rest of the sample, meaning that upweighting them sharply may not cause a massive distortion.  But an unwise weighting scheme still introduces a degree of uncertainty into proceedings that we could well do without.