Tuesday, March 9, 2021

The #Matchettgate fake poll scandal: what have we learned?

As you may have seen last night, Business for Scotland published an interview with a Savanta ComRes research director in an attempt to get to the bottom of Scotland on Sunday's decision to publish poll numbers that hadn't been turnout weighted in the normal fashion, and which were then parrotted without any disclaimer by Sarah Smith and Sophy Ridge on the Sunday morning politics shows.  The ComRes explanation is quite hard to follow and in one sense appears to be contradictory - initially it's suggested that the normal weighting couldn't be applied because of the shortage of time (it was a snap poll to capture the impact of Nicola Sturgeon's evidence to the Holyrood inquiry), but then it turns out that it would have been impossible to apply the weighting because respondents weren't even asked the likelihood-to-vote question, and the reason they weren't asked was that it had already been decided not to apply the weighting and therefore the information wasn't needed.  That doesn't make any sense unless ComRes have made a general decision to abandon turnout weighting for their independence polling, and they clearly haven't done that.  Quite the reverse, in fact - the explanatory note on the ComRes website yesterday made clear that the numbers published by Scotland on Sunday shouldn't be treated as a proper independence poll because of the lack of turnout weighting.

When an explanation doesn't entirely make sense, it's likely that there's a crucial piece of the jigsaw missing, ie. something we haven't been told.  There are at least two possibilities that are consistent with the known facts -

1) This was never meant to be a Yes/No independence poll, and therefore the turnout question wasn't needed.  The independence question wasn't intended for publication, but was nevertheless asked to provide more detailed data on the other questions - for example, to show what percentage of Yes voters had lost trust in Nicola Sturgeon, etc, etc.  The unweighted indy numbers appeared in the datasets as a matter of routine, but Scotland on Sunday got excited when they saw the No lead and insisted on publishing them.  ComRes were understandably reluctant to denounce their client, so instead a compromise was dreamed up - Scotland on Sunday would publish a disclaimer in the small print about the numbers not being directly comparable to other polls, while ComRes themselves would make clear that the numbers were merely an "indication of current Yes/No support" and not a proper independence poll.

2) The cock-up theory.  Even pollsters aren't infallible, so it's not impossible that the turnout question was left out by mistake.  Scotland on Sunday had been promised an independence poll by a certain time, so a decision was taken to plough on regardless, but to then make clear that it wasn't a proper independence poll.

In the Business for Scotland piece, Gordon MacIntyre-Kemp suggests that the turnout question may actually have been asked but was then left out of the data at the request of Scotland on Sunday.  That's possible, but my guess is that it's less probable than the above two explanations, because if Scotland on Sunday had played silly buggers to such an extreme extent, it's much more likely that ComRes would have drawn a line at that point and insisted on the correct voting intention numbers being used.

UPDATE: Having only just written this blogpost, I'm now going to have to undermine its central conclusion, because Gordon MacIntyre-Kemp is right - there's one key point that simply doesn't add up.  A significant percentage of respondents are missing from the independence results in the ComRes datsets, and the most likely explanation for that is the use of the basic turnout filter (as opposed to turnout weighting) - ie. anyone who is less than 60% likely to turn out to vote was excluded.  But it wouldn't be possible to apply that filter unless the turnout question had been asked - and if it was, why is it missing from the datasets?  Why wasn't it used for turnout weighting?  Why are ComRes saying it wasn't asked?  We may never be entirely sure what's happened here.


  1. This comment has been removed by the author.

  2. They're trying to create a false reality to drip feed into the minds of the population that the NO side are the new majority, albeit a single poll and they're willing to cheat or distort to push their agenda. This will make it harder to trust opinion polls now

  3. Wait, are the polls not showing Yes rising?

    The latest panelbase shows Yes up on the above comres...

  4. Over 20 polls showing a Yes lead and 1 doubtful poll showing a tiny No lead and the media are all over it like a rash. social media went into melt down a few weeks ago when a sub-sample poll showed a huge dip in support for the SNP. they are panicking.

  5. I understand this is the same people that called the Scots judiciary the 'corrupt clown office' the other week?


    Alister Jack: Boris Johnson will use courts to stop Scottish independence vote

    BORIS Johnson is willing to launch legal action to stop a vote on Scottish independence, Alister Jack has said.

    The Scottish Secretary insisted the UK Government will reject any calls for indyref2, regardless of the result of the Holyrood election.

    1. That'll be Boris - Hammer of the Scots.
      We'll see.

    2. And of the Welsh, and of the N. Irish, including unionists.

      He's trashing the union on 4 fronts.

      5 if you include Gibraltar which is basically part of the EU now.

  6. Since it is fashionable to report polls without the LTV filter...

    Panelbase (all respondents):
    46% Yes
    45% No
    9% DK / WNV

  7. Thank god for scotgoespop.

    FWIW I think the britnat Scotsman knew what they were doing in spewing out this false poll result. The britnat media will follow suit. Meanwhile the SNP... does what about it exactly.

  8. It hasn't shaken my faith in Newspapers James as I had no faith in any of them to begin with. I have had no faith in any of them since before the 2014 Referendum. Quislings the lot of them.

  9. Makes you think, where would we be if we'd used the last 6 years educating and informing the Scottish people in the facts about the wealth of national resources we would have at our disposal with the full powers of independence.
    Instead we even seem unable to defuse and effectively rebut the £15m GERS claim.
    When, Oh when will an SNP MP/MSP simply reply Are you saying Scotland is too poor to be independent?, when the GERS question is raised.
    This has only happened once to my knowledge when an SNP MSP (Humza Yousaf)? pointed out Scotland's wealth and was applauded by a mainly English audience.
    The SNP should be rebutting all the crooked polls and Sarah Smith type bias from so called journalists.
    The SNP MP's can't surely be that busy in London. GIE ONE OF THEM A REAL JOB!

  10. What happened to the poll that was to be released yesterday?