Saturday, September 24, 2022

Sensational Social Attitudes Survey suggests that independence support has soared by almost TWENTY percentage points since the 2014 indyref - does this suggest that conventional polling has been underestimating Yes support for years?

I've finally had a chance to catch up with the independence numbers from the Social Attitudes Survey that caused so much excitement a couple of days ago.  What strikes me most is that the survey seems to exist in a slightly different universe from the more conventional polling we're used to seeing.  We thought we knew that 2020 was (by far) the high watermark for independence support to date, and that 2021 was a step backwards, but this new survey suggests that support actually reached a fresh peak in 2021 - albeit there was no Social Attitudes Survey in 2020 itself for obvious reasons, so it's possible an even bigger number was missed during the gap.  Nevertheless, an outright majority for independence on a multi-option question is a startling finding given the pattern of polls in 2021.  It could perhaps be more easily explained if the fieldwork had been conducted at the start of the year, because the unbroken sequence of Yes majority polls carried on into January 2021 (as I remember extremely well, because I commissioned a Survation poll myself that month which showed Yes on 51%).  But in fact the Social Attitudes Survey was conducted in the autumn, and was thus contemporaneous with a series of eight polls in a row which showed a No lead, albeit mostly a fairly modest No lead.

2021 Scottish Social Attitudes Survey constitutional preferences (with changes from 2019):

Independence: 52% (+1)
Devolution: 38% (+2)
No Scottish Parliament: 8% (+1)
Don't Know: 1% (-4)

The main difference of format between this survey and conventional polls is that respondents weren't asked to give a binary Yes/No answer to independence, but instead to choose between multiple options, including more than one option for expressing support for the union.  You would normally expect such a format to produce less support for independence than the conventional polls, and yet for some reason it's produced more.  

But, actually, I would tend to suspect that if a binary question had been asked in the survey, it might have resulted in an even bigger Yes vote.  I think there's something going on here that isn't about the question format - I think it might be about the data collection method.  The Social Attitudes Survey has traditionally been conducted face-to-face, which makes it very different from the online polls we're used to seeing.  If the 2021 survey had been face-to-face as usual, that would be a very obvious potential explanation for the divergence with conventional polls, and might call into question whether online polls have been systemically underestimating the Yes vote for some time - especially bearing in mind that the only telephone poll conducted in autumn 2021 (by Ipsos-Mori) also contradicted the online polls by showing a very handsome Yes lead.

But just to muddy the waters, the pandemic meant that the Social Attitudes Survey wasn't conducted face-to-face last year - the What Scotland Thinks website suggests it was a hybrid telephone/online panel survey.  Nevertheless, I do still wonder if the telephone element, and perhaps a different approach to online fieldwork (particularly random participant selection), may have been factors in producing such a strikingly different result from conventional polls.

The other sense in which the Social Attitudes Survey seems to exist in a different universe is the longer term trend.  It suggests that support for independence has more than doubled since 2012, has increased by almost twenty percentage points since the indyref year of 2014, and has increased by thirteen points even since the SNP's golden year of 2015.  There's no sign in conventional polling of quite such a dramatic transformation, and Stuart Campbell must be looking on in horror, because it's a trend that's utterly impossible to reconcile with his repeated claims of flatlining Yes support since 2014.  The question format probably is the explanation here, because it looks as if a lot of people who said they supported devolution in past surveys nevertheless voted for full independence when presented with a binary choice in a referendum.  But their increasing readiness to actually identify as independence supporters rather than devolution supporters suggests that their position has hardened considerably as the years have progressed.

And just a final thought.  Renowned Liberal Democrat "fast bowler" Alex Cole-Hamilton tried to mock The National for reporting a survey with fieldwork that was a year old.  "Whatever this is, it's not journalism" he sneered.  But hang on.  Isn't it his own side that claims that the state of public opinion eight years ago is more important than the state of public opinion now?  Even though in 2014 nobody knew Brexit was coming, and even though some people who are now eligible to vote were only eight years old back then?

*  *  *


If you've donated already, a million thanks!  And if you'd like to make a donation now, here are the various options...

Direct payments via Paypal - my Paypal email address is:

(Paypal payments are the best method because they're direct and eliminate all fees as long as you choose the "paying a friend" option.  However, please take great care to spell the above email address correctly.  Also, if you wish you can add a note saying something like "for the fundraiser", but rest assured it'll be obvious what the payment is for anyway.)

Scot Goes Pop General Fundraiser 

Scot Goes Pop Polling Fundraiser 

If you prefer a bank transfer, please message me for details using the contact email address which can be found in the sidebar of the blog (desktop version only), or on my Twitter profile.


  1. James, please set up a Patreon so people can contribute on a regular basis, if they choose. If you decide to give it up blogging at some point, then you can easily close the Patreon, but in the meantime, and given your long track record of consistent output, it’s a good way to allow people to support the blog in the long term

  2. James, a Youtube channel with weekly (or even daily) original content is a way to professionally monetise your efforts. A blog is fine for those who've found you but won't promote you outside the existing pool of 'clever' folks who read your work. The quicker you start the better.

    1. I hear what you say, but it's not straightforward. One of the most recent YouTube videos I did was the interview with Lisa Keogh during the council elections. I had to make a few minor edits, and it took me absolutely ages. A regular YouTube channel could end up being so time-consuming that I'd have no time left for the blog.

    2. Doesn't have to be a big interview, just 10 minutes giving your opinion on any pertinent subject but keeping your polling analysis as your principal superpower. You'd need a decent webcam and microphone, a 3 to 5 second introductory mucic theme. Some Youtube channels I go on are :

  3. Bloody Hell - Labour singing the national anthem and union flag logos pasted everywhere. Labour = Tory lite.

    Cole - Hamilton is the Lib Dem Scottish Leader and MSP who served on the Scottish Parliament Inquiry into Sturgeons persecution of Salmond whilst keeping secret the fact that he maintained a secret relationship with one of the original two alphabetties. Whatever he is he is not a person with any integrity.

    The UK is a shithouse with shitty politicians in shitty political parties.

  4. New poster here. I just wanted to comment about the SSA survey which I took part in. It was a phone call, about an hour long, but it wasn't autumn 2021, it was March 2022.

    1. Are you sure it was the same survey? The website gives the dates as September and October 2021.

    2. 100%. I got this email from ScotCen shortly after the phone call.
      "Thank you for your help

      Dear xxxxx,

      Thank you for recently completing the Scottish Social Attitudes survey. We are grateful to you for taking the time to do this.

      As a thank you for your time, here are the details of your £10 Love2shop voucher.," etc etc

      ...also said "results will be published in 2022" and given it was already 2022 I'm guessing they ran over by a few months!

  5. I agree that Patreon might be a guid gate tae gang. I support a couple of people on there and would certainly add James to my subscriptions.