False alarm: Panelbase haven't stopped using the 2014 referendum question

So just to briefly tie up a loose end from the blogpost on Sunday, it turns out that Britain Elects were wrong (gasp!) and in fact Panelbase did not alter the wording of the question they asked about independence for the Sunday Times poll - it was the usual question, with the 2014 referendum wording.  So that's good news - it means that a BBC journalist's criticism of the SNP for "promoting" a poll with a non-Yes/No question was based on a false premise, and it also means that the pro-Yes trend in the poll hasn't been called into question by methodological changes.

In a nutshell: false alarm.

*  *  *

I've just caught up with yet another polling post on Wings Over Scotland, and I really do have to reiterate that it's deeply reprehensible for him to be seeking to pass the blame onto Panelbase for his own biased and leading questions about the trans issue (and indeed about other issues).  Here's what he said this time -

"Some of those polls have been conducted by this site, and are invariably leapt on by trans activists as having featured supposedly “leading” or “unfair” questions, even though we check all our questions with Panelbase first and the results have without exception been identical, often to within a single percentage point, to those revealed in polls featuring differently-worded questions commissioned by extremely trans-friendly organisations like Pink News."

If I thought for one moment that he'd asked Panelbase "are these questions leading?" and that they had said "no", then I would be criticising Panelbase and saying they had to accept a share of the blame.  But I just don't believe that's what's happened here.  This is the question he published a few days ago -

"The SNP has announced its intention to implement 'self-ID' legislation, whereby physically-male people will have unrestricted access to all female-only spaces and services (eg. toilets, hospital wards, changing rooms, sporting competitions and women's refuges) if they declare themselves to be women, whether or not they've had any medical treatment or surgery to change their sex.

On a scale of 0 to 10, how do you feel about this proposal?"

It's absolutely fatuous to suggest that any polling company would have said the above question is even-handed or neutral, and yet Mr Campbell is trying to give us the impression that Panelbase did exactly that.  In the comments section of this blog, he even provided a direct quote from an email sent to him by his Panelbase contact to try to bolster that impression.  Given how emotive this subject is, I would imagine that Panelbase are not exactly over the moon that one of their clients is effectively telling people that they've picked a side in the trans debate.  That's the inescapable implication of what Mr Campbell is saying, because anyone suggesting that his question is neutral would undoubtedly be picking a side.

My very strong suspicion is that Panelbase were answering a completely different query - I suspect they were asked if Mr Campbell's question was acceptable, and they said it was.  Acceptability and even-handedness are two entirely different things. Every day of the week, polling companies ask questions on behalf of paying clients that are calibrated to produce the best possible results for that client from a presentational point of view.  The threshold for saying "no" to a paying client is extremely high - but one reason for doing so might be that there are factual inaccuracies in the way that the question is worded.  Panelbase would certainly have been able to confirm to Mr Campbell that there were no inaccuracies in his question, and that it was therefore within the bounds of acceptability.  For Mr Campbell to portray that reply as some sort of official seal of approval, and as proof that his question is non-leading and even-handed, is deeply disingenuous and downright cynical.

He goes on to make the point that the results of his poll are similar to others with different wordings.  That's something I've noticed myself, but it doesn't give him a free pass.  Not all polls with leading questions produce misleading results, but that doesn't magically mean that the questions are all fine.

Today he's published new polling that finds only 25% of respondents think people should be able to change the sex on their birth certificate.  He insists that the wording of this latest question wasn't remotely leading.  That's fine - I actually accept that.  But I would just make the point that "Do you think that people should be able to legally change gender?" is a somewhat different question from the one he asked, and that people who answer "yes" on that point may also take a different view on whether an individual who has gone through the necessary legal process should then be able to seek an amendment to their birth certificate.  Mr Campbell's question is more legitimate than his previous ones and it does tell us something interesting, but it doesn't tell us everything.  Some of the people answering his question may not even have been taking the existence of trans people into account, and may have just answered on the basis that an individual "shouldn't be able to falsify details on their birth certificate" without realising how their responses would be interpreted.  I rather suspect that was Mr Campbell's intention, given that he was also asking people at the same time whether individuals should be able to change their date and place of birth to something false.  (Indeed the specific question asked was "which of these facts do you think people should be able to change on their birth certificate if they want to?", with the word "facts" implying that any change would be a departure from accuracy.)  So not a leading question as such, but a potentially deceptive question in a slightly different way.

And remember I say all of this as someone who is broadly opposed to the proposal on self-ID.  Propaganda polls just make me bristle, even when they're commissioned by people on my own side.