The #Matchettgate fake poll scandal: what have we learned?

As you may have seen last night, Business for Scotland published an interview with a Savanta ComRes research director in an attempt to get to the bottom of Scotland on Sunday's decision to publish poll numbers that hadn't been turnout weighted in the normal fashion, and which were then parrotted without any disclaimer by Sarah Smith and Sophy Ridge on the Sunday morning politics shows.  The ComRes explanation is quite hard to follow and in one sense appears to be contradictory - initially it's suggested that the normal weighting couldn't be applied because of the shortage of time (it was a snap poll to capture the impact of Nicola Sturgeon's evidence to the Holyrood inquiry), but then it turns out that it would have been impossible to apply the weighting because respondents weren't even asked the likelihood-to-vote question, and the reason they weren't asked was that it had already been decided not to apply the weighting and therefore the information wasn't needed.  That doesn't make any sense unless ComRes have made a general decision to abandon turnout weighting for their independence polling, and they clearly haven't done that.  Quite the reverse, in fact - the explanatory note on the ComRes website yesterday made clear that the numbers published by Scotland on Sunday shouldn't be treated as a proper independence poll because of the lack of turnout weighting.

When an explanation doesn't entirely make sense, it's likely that there's a crucial piece of the jigsaw missing, ie. something we haven't been told.  There are at least two possibilities that are consistent with the known facts -

1) This was never meant to be a Yes/No independence poll, and therefore the turnout question wasn't needed.  The independence question wasn't intended for publication, but was nevertheless asked to provide more detailed data on the other questions - for example, to show what percentage of Yes voters had lost trust in Nicola Sturgeon, etc, etc.  The unweighted indy numbers appeared in the datasets as a matter of routine, but Scotland on Sunday got excited when they saw the No lead and insisted on publishing them.  ComRes were understandably reluctant to denounce their client, so instead a compromise was dreamed up - Scotland on Sunday would publish a disclaimer in the small print about the numbers not being directly comparable to other polls, while ComRes themselves would make clear that the numbers were merely an "indication of current Yes/No support" and not a proper independence poll.

2) The cock-up theory.  Even pollsters aren't infallible, so it's not impossible that the turnout question was left out by mistake.  Scotland on Sunday had been promised an independence poll by a certain time, so a decision was taken to plough on regardless, but to then make clear that it wasn't a proper independence poll.

In the Business for Scotland piece, Gordon MacIntyre-Kemp suggests that the turnout question may actually have been asked but was then left out of the data at the request of Scotland on Sunday.  That's possible, but my guess is that it's less probable than the above two explanations, because if Scotland on Sunday had played silly buggers to such an extreme extent, it's much more likely that ComRes would have drawn a line at that point and insisted on the correct voting intention numbers being used.

UPDATE: Having only just written this blogpost, I'm now going to have to undermine its central conclusion, because Gordon MacIntyre-Kemp is right - there's one key point that simply doesn't add up.  A significant percentage of respondents are missing from the independence results in the ComRes datsets, and the most likely explanation for that is the use of the basic turnout filter (as opposed to turnout weighting) - ie. anyone who is less than 60% likely to turn out to vote was excluded.  But it wouldn't be possible to apply that filter unless the turnout question had been asked - and if it was, why is it missing from the datasets?  Why wasn't it used for turnout weighting?  Why are ComRes saying it wasn't asked?  We may never be entirely sure what's happened here.

POLL: Has the #Matchettgate fake poll scandal shaken your faith in newspapers and broadcasters as reliable sources?

— James Kelly (@JamesKelly) March 8, 2021

So I've been watching the Harry & Meghan thing on the +1 channel, and I thought to myself "they're both so charismatic and convincing, this is the British establishment's worst nightmare".

— James Kelly (@JamesKelly) March 8, 2021

And then I thought "God, how the Royals must wish they had a stern Nicholas Witchell to rebut the beastly accusations in real time, on a split screen". And then I burst into uncontrollable laughter.

— James Kelly (@JamesKelly) March 8, 2021