Friday, October 11, 2019

Disappointing but not remotely surprising: Stuart Campbell uses ludicrously vague "Archie Stirling"-type polling question to create the false impression of potential support for a Wings party

So Stuart Campbell's interminable drip-drip reveal of the details of his propaganda poll has finally reached the culmination that we all knew was coming.  There are no surprises here - a commenter on this blog had warned us several days ago that Mr Campbell had asked a dodgy "Archie Stirling"-type question in the poll in order to generate the false impression that there was significant potential support for the new political party that he hopes to set up in direct competition with the SNP.

What is an Archie Stirling-type question?  Just weeks before the 2007 Holyrood election, the wealthy businessman Archie Stirling (ex-husband of Diana Rigg and father of Rachael Stirling) commissioned a YouGov poll which asked respondents whether they would consider voting for his new centre-right political party, Scottish Voice, on the regional list ballot.  21% said they would.  Mr Stirling sent the results to the newspapers, which breathlessly reported that Scottish Voice could be on course to win dozens of list seats and to hold the balance of power.  But a few weeks later when the actual results came in, the party received only 0.3% of the list vote and didn't come remotely close to winning a single seat.  It had won just one-seventieth (!) of the number of votes that the YouGov poll had implied was possible.

History repeated itself earlier this year when Change UK was set up.  A number of polls asked whether voters would consider voting for the new kid on the block, and the results suggested astronomical levels of support.  But when the European elections came around, Change UK took 3.3% of the vote and no seats.  (At least that was a more respectable result than Archie managed.)

Why does the "would you consider voting for...?" question produce such wildly misleading results?  It's just basic human psychology.  If you ask me whether I'll consider eating a banana, I'll say "yes, of course I'd consider it, I have nothing against bananas".  But if you then tell me I have to choose just one piece of food for my next snack and ask me whether it'll be a banana, or crisps, or pizza, or yoghurt, or spaghetti, or a toastie...well, it's considerably less likely that the banana will get the nod.  In a similar way, voters will have just one vote in the regional list ballot in 2021, and to get a meaningful sense of how well Wings might fare you'd have to present the party as merely one of a menu of options.  The question that would have cleared the mists is as follows...

If the following parties stand on the Scottish regional list ballot in 2021, which one would you vote for?

Scottish National Party (SNP)
Liberal Democrats
Brexit Party
Wings Over Scotland
Scottish Socialist Party (SSP)

Mr Campbell has no excuses for not knowing that was the type of question that needed to be asked.  It's not just me that's been saying this for weeks - others have made the exactly the same point in comments on his own site, and we know he saw those comments because he directly responded to some of them.  The fact that he's gone right ahead and asked the Archie Stirling-type question anyway means we're entitled to conclude that those who trusted him to use his polling as a genuine attempt to accurately measure support were wrong.  This is instead a propaganda exercise to justify a decision that has to all intents and purposes already been taken.

Why does this matter?  Because if, as seems overwhelmingly likely, the Wings party fails to take at least 5% of the vote in at least one electoral region, it will not win any seats at all, and any votes it does take will make it harder for larger pro-indy parties (ie. the SNP and the Greens) to win list seats.  If a credible poll had been conducted showing that Wings was on 1% or 2% or 3% or even 4% of the list vote, the pressure on him to do the sensible thing and drop the whole idea would have been overwhelming, even from some of his own supporters.  And he couldn't risk that, could he?  So instead he's asked a dodgy question that he knew would produce meaningless results that he could spin any way he likes.  And by God is he spinning - he's even making the barking mad implication that the poll shows that six out of seven SNP voters might make the jump to his party.

Incredibly, the question he asked is actually a hundred times worse than Archie Stirling's question was.  At least Mr Stirling had the basic decency to actually mention the name of the party he was polling about.  Mr Campbell's question doesn't even do that.

While still voting SNP with your constituency vote, would you be prepared to consider voting for a new pro-independence party with your list vote, with the intent of increasing the number of pro-independence MSPs in Parliament?

Where do you even begin with the nonsense of that question?  Given Mr Campbell's supposed pride in the brand recognition of Wings, why would he be so shy about actually identifying this "new pro-independence party" as the Wings party?  It can only be because he thought he'd get a more favourable result by being as vague as possible and making it sound as if he might be talking about an entirely different sort of pro-independence party.  It's the ultimate 'catch-all' polling question.  And why on earth didn't he end the question with the words "with your list vote?", rather than adding the unnecessary leading wording about "increasing the number of pro-independence MSPs", which was bound to make it harder for respondents to give a negative reply?  Well, quite, I've answered my own question there.  The results would probably have been very different without those words, which are a) downright misleading, and b) superfluous to the issue of whether voters are actually looking for an alternative to the SNP, let alone an alternative called Wings.  If I asked you "Would you be prepared to consider changing jobs with the intent of getting more money?", it would be hard to say "no", but what would I actually be proving?  I wouldn't be surprised if some respondents gained the false impression that they were being asked what they would "consider" doing if a new pro-indy party stood on the list ballot with the SNP's blessing.  The question does read as if it's implying some sort of masterplan on behalf of the wider Yes movement.

I would go so far as to say the results of such a ridiculous question should be regarded as completely worthless, but for the sheer hell of it I'll try to make some sense of them.  19% of respondents say they would "definitely consider" (whatever that means) voting for this unspecific new party.  Remember that we're only talking about SNP constituency voters here, so that's probably only around 8% or so of the total electorate.  If I was cheeky enough to suggest that history will repeat itself from 2007 and that only one-seventieth of those people will actually end up voting for Wings, that means Mr Campbell is on course for a humiliating 0.1% of the list vote.  Oh, but then of course there's the additional 56% who say they would "perhaps" consider voting for the party "depending on its policies".  Well, no shit, Sherlock.  Voters who will decide whether or not to "consider" voting for a party when they actually find out what it is and what its policies are?  HOLD THE FRONT PAGE.

This is an insult to the intelligence of people who have been giving Mr Campbell the benefit of the doubt over recent weeks, and they have every right to be angry with him.  Doubtless there will still be many Wings readers who are wide-eyed enough to take the preposterous "six out of seven" claim at face value, and they're going to be in for a nasty shock one of these days - because sooner or later a media organisation will run a credible poll about the Wings party along the lines that I suggested above.  Then we'll see the true picture, and there'll be no hiding place left. (Although probably a few people will be so deep into the trance by that point that they'll accuse whoever ran the poll of "rigging" it.)

I know that some people will innocently protest that Mr Campbell says in his write-up that he plans to ask more "specific" polling questions on the subject himself in future, but come on.  Let's get real.  If he was remotely serious about doing that in timely fashion, he would have done it in this poll.  I defy anyone to come up with a plausible explanation for why he hasn't.

Incidentally, any time Mr Campbell has been challenged on the series of absurdly leading questions he's asked in this poll, he's come up with the rather weak stock reply of: "Take it up with Panelbase, they okayed it, and here's an email they sent me as proof."  See for example this exchange of last night and today.  To avoid having to repeat the same thing another 57 times, here's what I've been saying in reply:

"No, I won't do that. What polling companies do to make money is ask questions that clients want asked in return for thousands of pounds. They tend to be pleasant and accommodating to those clients for entirely understandable reasons, regardless of the agenda that the client is pushing. That can be seen, for example, in Survation's willingness to ask a certain 'voting intention' question on behalf of Scotland in Union that I doubt if any of us - including you - consider to be a genuine attempt to measure public opinion on independence.

He who pays the piper calls the tune. I'm far more interested in holding the person who calls the tune accountable, rather than some unnamed person from Panelbase who may have fallen into your little trap of saying something unwise when they were trying to please a paying client and when they presumably thought they were speaking in confidence.

Basically your complaint here is that your polling is being regularly analysed in a polling blog. With all due respect, it's hard to think of a more fatuous and futile complaint than that. If being mentioned in a polling blog really bothers you so much, all I can suggest is that you stop publishing polling results on an almost daily basis. Alternatively you could just chill out and accept that when you step into the political arena in a free society, people have the right to comment on your decisions and actions."

YouGov average suggests SNP remain on course for sweeping gains from both the Tories and Labour

Regular readers will recall that I became more than a little concerned last week after a third YouGov subsample in a row put the SNP's support at an unusually low level.  The results of YouGov's Scottish subsamples tend to be much more stable than those from other firms, because YouGov appear to structure and weight their subsample figures correctly.  Three low results in a row could still have happened by random chance, but the worry was that if a fourth and fifth in a row showed the same thing, it would start to look very much like something had genuinely changed.

I'm pleased to be able to report that normal service has been resumed in the newest YouGov subsample out today.

SNP 43%, Conservatives 20%, Brexit Party 10%, Labour 10%, Liberal Democrats 8%, Greens 6%

In conjunction with the results from other pollsters that don't show any obvious recent dip in support for the SNP, I'm inclined to think this probably means that the run of bad results was just caused by freakish sampling variation and that SNP support has actually been holding reasonably steady.  But obviously we'll keep an eye on it.

What's particularly encouraging is the poor showing for the Liberal Democrats in the latest subsample - in the earlier results it had looked like it was the Lib Dems that were doing some of the apparent damage to the SNP.

Given the small number of respondents interviewed for each subsample, the best way of getting a meaningful sense of the state of play is to average several subsamples over time.  Here is the average of the last five from YouGov -

SNP 39.2%
Conservatives 21.0%
Liberal Democrats 14.0%
Labour 12.2%
Brexit Party 6.6%
Greens 5.6%

For what it's worth, the Electoral Calculus model suggests that would give the SNP a net gain of 13 seats and leave them with 48.  The Tories would lose 8 seats, although annoyingly that would still leave them with 5, and the rump group would include the likes of David Mundell and Ross Thomson.  (Perhaps we should be a tad sceptical about the latter given that his nickname at Westminster has long been "SNP gain".)  Inevitably it's Labour that would take the absolute hammering, with Ian Murray the last man standing once again.  The only SNP loss would be to the Liberal Democrats in the ultra-marginal seat of North-East Fife.

Of course what we could really do with now is a full-scale Scottish poll just to confirm that the above numbers are in the right ball-park.  Given that a general election is probably only a few weeks away (a few months at the absolute most) and that we're in the midst of the biggest national crisis since the Second World War, it's extraordinary that we haven't had a full-scale poll for over a month.

Thursday, October 10, 2019

Guinness Book of Records on standby as latest Wings poll question is even more loaded than the previous ones

There were many silly moments in the immediate aftermath of Mr Campbell throwing his toys out of the pram in early September because I refused to pipe down about the inherent problems with the idea of setting up his own political party, but the silliest one of all came when one of his supporters donned a deerstalker and unearthed a tweet in which I'd suggested that a Wings poll had once asked an absurdly leading question about the trans self-ID issue.  "I think there may be more to Mr Kelly's opposition than meets the eye" said the Wings supporter (or words to that effect) to which Mr Campbell replied "Aha!" as if he'd just found incontrovertible proof that I only opposed the Wings party because I was a secret supporter of self-ID.

The reason this was monumentally silly is, of course, that I'm not a secret supporter of self-ID.  Quite the reverse.  I've been on the record for months in saying that I broadly agree with Mr Campbell on the trans issue.  It certainly doesn't rank as highly on my list of priorities as it does for him, and I think it always rings a bit phoney when he couches his views on the subject in the language of radical feminism, but nevertheless I do wish the SNP would take their foot off the accelerator on self-ID and at the very least seek a meaningful compromise.  I don't think this is the ditch they should be dying in.

At the end of the day, though, a leading polling question is a leading polling question, and Mr Campbell has just repeated the exercise in his latest poll (conducted among SNP voters only).  I defy anyone to say with a straight face that the following wording can be regarded as neutral.

"The SNP has announced its intention to implement 'self-ID' legislation, whereby physically-male people will have unrestricted access to all female-only spaces and services (eg. toilets, hospital wards, changing rooms, sporting competitions and women's refuges) if they declare themselves to be women, whether or not they've had any medical treatment or surgery to change their sex.

On a scale of 0 to 10, how do you feel about this proposal?"

When I pointed out the leading nature of the previous similar question, it was suggested to me that the lack of public knowledge about the issue means that you'd only get meaningful results if you explain the implications of the policy when asking the question.  That's fine, but if you're going to explain the issue you actually have to do it in an even-handed way (unless of course the purpose of the poll is propaganda rather than the accurate measurement of public opinion).  If it's not possible to find genuinely neutral language on such an emotive topic, one option would be to explain how proponents see the proposal and then counterbalance that with how opponents see it.  Essentially Mr Campbell's question gives respondents the case for the prosecution but not for the defence.  All of the main concerns of the anti-self-ID lobby are carefully itemised (including the mention of women's refuges, which is calculated to produce a certain reaction), but that process is not repeated for the concerns of the pro-self-ID lobby.  There's also a tone of incredulity throughout - trans women are not trans women but "physically-male people" and they will not merely have access to female-only spaces, but "unrestricted" access, and they'll have it "whether or not" they've "change[d] their sex".  No-one can accuse Mr Campbell of sparing the kitchen sink in this question.

My own guess is that the results of the poll might not have been all that radically different if a fairer question had been asked, and if I'm right, Mr Campbell has undermined the credibility of the numbers for no good reason.  It's all a bit pointless.

Still no sign of the dodgy question we all know he asked about the Wings party idea.  Maybe I should open a book on when (if ever) he'll get round to publishing those results.

Tuesday, October 8, 2019

It's actually pretty remarkable that two-thirds of SNP voters expect a pre-2021 indyref

The drip-feed of results from the new Wings poll continues, but there's still no sign of the question that everyone thinks is coming.  (Maybe even before Christmas.)  In the meantime, Mr Campbell is taunting his critics with a result that he claims they won't like, ie. that 'only' two-thirds of SNP voters expect there to be an independence referendum before May 2021.  Frankly, that's a remarkably upbeat finding and all I can say is that I hope these people know something I don't.  I certainly haven't given up hope on a pre-2021 referendum, but given Nicola Sturgeon's apparent determination that the Section 30 process should be repeated, it's going to be like threading a needle.  The forthcoming election would probably need to produce a minority Labour government with the SNP holding the balance of power, and that's not an outcome that the SNP can contrive - it'll either happen by chance or it won't.

Incidentally, Mr Campbell uses an age-old trick to misrepresent the result of the poll - he combines the 21% of respondents who don't expect a pre-2021 referendum with those who said "Don't Know", in order to claim that "a third of SNP voters" are "unconvinced" by the First Minister's assurances that an early referendum is coming.  It would be just as easy to use the Don't Knows in the opposite way and claim that "79% of SNP voters don't share Mr Campbell's scepticism".  But a much fairer and more meaningful thing to do is simply to strip out the Don't Knows altogether, which would leave us with approximately 76% who anticipate a pre-2021 vote.

The second finding that Mr Campbell has announced with misplaced triumphalism is that 57% of respondents agree in principle with his cunning plan that the SNP should facilitate Brexit in return for the permanent transfer to the Scottish Parliament of the power to call independence referendums.  I'd have agreed to that proposition myself if I'd been answering the poll, but Mr Campbell is making the same fundamental mistake that the UK government has been making in the Brexit negotiations for the last three years - he thinks that a deal is something that you make with yourself.  Here's the snag: it wouldn't matter if the SNP were willing to cut a deal, because the Tory government are not (and never have been) remotely interested in agreeing to the required terms.  And in a way there's a degree of logic to that, because diehard unionists in the Tories and the DUP would probably walk away in disgust from any pro-Brexit coalition that included the SNP, on the grounds that it would "weaken the union".  It may well be that SNP votes wouldn't actually be able to deliver Brexit.

The SNP group in the Commons simply doesn't have the potential leverage with the Tories that Mr Campbell believes.  So what's Plan B, wise guy?

Wings poll: Disagreement is not the same thing as "not having a clue"

As a number of people feared, the latest Panelbase poll commissioned by Wings Over Scotland appears to be largely a propaganda exercise to further that site's recent anti-SNP agenda, and in particular Mr Campbell's tentative plans to set up a new political party in direct competition with the SNP.  Only people planning to vote SNP on the Holyrood constituency ballot were interviewed.  The idea seems to have been to identify a series of ways in which the SNP are regarded as deficient by their own voters, and then to present a magical 'solution' to these supposed 'problems'.

The results from the poll are being drip-fed to us, and the latest batch purport to show that SNP voters are confused by the party's strategy and aims in regards to Brexit.  Actually the poll shows no such thing.  It shows that SNP voters disagree with each other in their interpretations of what the strategy and aims are, but there's no evidence at all that individuals are unsure in their own minds.  (To get that sort of evidence, there would have had to be a question along the lines of "On a scale of 0 to 10, how confident are you that you understand the SNP's goal in relation to Brexit?")  Even the supposed division of opinion among SNP voters is somewhat artificial, because a clear majority of respondents (56%) said that they thought the aim was either to stop Brexit altogether for the whole UK, or to stop a No Deal Brexit for the whole UK.  The difference between those two options is one of emphasis, because stopping No Deal probably requires stopping Brexit altogether, at least for the time being.

It's true that another 26% of the sample do hold an interpretation that contradicts the view of the majority, ie. they think the SNP want a Scotland-specific Brexit deal within the UK.  But that's a product of the evolution of the SNP's own position - at one point Nicola Sturgeon was pressing for exactly such a deal, but having failed to get blood out of a stone she started taking the view that the only way (short of independence) to keep Scotland in the single market and/or the EU itself was to keep the whole of the UK in.

What I find really encouraging from the results is that only a very small 8% of the sample take the cynical view that the SNP are just "pretending" to try to stop Brexit as a way to win voters over to the independence cause.  That's been one of the Scottish Tories' favourite smears ("the SNP are desperate for No Deal"), and it appears from the poll that it simply hasn't resonated with SNP voters.  But what would be more helpful to know is whether it's resonated with the wider electorate.

Several Wings supporters have indignantly protested to me in recent weeks that Mr Campbell is extremely hard-headed, and would use his forthcoming polling as a genuine attempt to measure the likely support for a Wings party.  If that support wasn't there, he would drop the whole idea to avoid splitting the list vote and possibly reducing the number of pro-independence MSPs.  But what would a genuine attempt to measure support for a Wings party look like?  It would have to be a question that presented Wings as one of a menu of options.  Something along the lines of...

If the following parties stood on the Scottish Parliament regional list ballot, which one would you vote for?

Scottish National Party (SNP)
Liberal Democrats
Brexit Party
Wings Over Scotland
Change UK
Scottish Socialist Party (SSP)

If the Wings party scored significant support on a question of that sort (assuming, of course, that it wasn't at the end of a leading 'question ladder'), Mr Campbell would be entitled to say that he's onto something.

What a legitimate question wouldn't do is ask about a Wings party in isolation, ie. "Would you consider giving your regional list vote to a Wings Over Scotland party?", because history shows that a question of that sort will give a wildly distorted impression of how well a party might do.  The classic example was the YouGov poll that Archie Stirling commissioned when he set up his Scottish Voice party in 2007 - he managed to breathlessly persuade newspapers to report that polling showed his party was on course to take 20% of the vote and dozens of list seats.  Just weeks later, Scottish Voice took a mere 0.3% of the list vote and naturally didn't come within light-years of winning any seats at all.

Similar dodgy polling questions earlier this year gave the bogus impression that the Independent Group/Change UK was on course for a massive breakthrough at Westminster.

So if Mr Campbell asks a credible polling question that presents Wings as one of a menu of options, it can reasonably be inferred that he is serious about accurately measuring support, and if he asks a vaguer question about the Wings party in isolation, it can reasonably be inferred that his motivation is somewhat different.  And guess what?  It looks like he's done the latter.  An anonymous commenter on this blog was interviewed for the poll, and recalled that the question asked was along the lines of -

"Would you consider voting for a new alternative pro-independence party on the list ballot?"

Mr Campbell responded to the comment in his trademark derisive and foul-mouthed manner, but what he very noticeably didn't do was deny that he'd asked that sort of question.  He isn't a fool - he knows exactly what he's doing.  Even if at one point he was naive enough to believe that such a vague question would produce meaningful results, it's been pointed out to him for weeks why that isn't the case.  I wrote a blogpost in mid-August setting out the problem with Archie Stirling-type questions, and Alex Birnie has repeated the point several times in comments on Wings itself.  We know that Mr Campbell saw at least some of those comments, because he directly responded to them.

If, as it appears, he's gone ahead and asked that question anyway, it's safe to conclude that this is not an attempt to accurately measure potential Wings support on the list ballot.  It's most likely an attempt to generate the impression of significant support, to whip his most devoted fans up into an even greater frenzy, and to make the momentum towards setting up the party unstoppable, regardless of the damage it might do to pro-independence representation in the Scottish Parliament.  Remember that a small party that takes less than 5% of the list vote in any region will almost certainly not take a seat in that region, and can only do damage by taking votes away from larger pro-indy parties that do have a chance of winning list seats.

My advice to Wings readers, regardless of whether you're sympathetic to the idea of a new party or not, is this: demand better polling.

Sunday, October 6, 2019

Better news for the SNP with Opinium

As a counterbalance to my post about the run of relatively unimpressive YouGov subsamples for the SNP, here's some better news from Opinium (well, "better" if you leave aside the enormous Tory lead at GB level).

Britain-wide voting intentions (Opinium):

Conservatives 38% (+2)
Labour 23% (-1)
Liberal Democrats 15% (-5)
Brexit Party 12% (+1)
SNP 5% (n/c)
Greens 4% (+2)
UKIP 1% (+1)

Scottish subsample: SNP 48%, Conservatives 25%, Labour 11%, Brexit Party 9%, Liberal Democrats 7%

The subsample figures shouldn't be taken as seriously as YouGov's, because Opinium don't properly structure/weight their Scottish subsamples.  Nevertheless, we can certainly draw some reassurance from the fact that this is the second Opinium poll in the space of a week to put the SNP on a hefty 5% of the Britain-wide vote, and with a bit of luck that might mean that the SNP's recent dip in YouGov polling has been caused by random sampling variation rather than by real changes on the ground.  Time will tell.

No other pollster has yet picked up the slump in Lib Dem support across Britain that Opinium are reporting, so we should take that finding with a pinch of salt until it's corroborated.  But at the very least, it looks unlikely that the Lib Dems have been making any more progress over the last week.

There's also no cause for panic about the 15-point Tory lead across Britain, because Opinium have recently emerged as the most Tory-friendly pollster, producing figures for the Conservatives that are several points higher than other firms.  That doesn't necessarily mean they're wrong, of course, but there's a degree of uncertainty about whether the Tories are really that far ahead.  In any case, we know that the electorate is extremely volatile at the moment, and that the situation can change very quickly.  In particular, I'd have thought that Remain voters in England may well coalesce mainly around one party as the official campaign progresses - that party might be Labour or it might be the Lib Dems, but at some point people are going to stop looking for perfection, and start looking for the best option to stop Boris Johnson.