So just a quick postscript to the previous post about the ludicrous misreporting of the independence figures from the Progress Scotland poll. I was taking a peek at Conservative Home yesterday, and I stumbled across an article from the chap who mans their Colonial Desk (ie. he writes an occasional round-up of what might loosely be described as "news" from Scotland, Wales and Northern Ireland so that no-one else has to acknowledge the existence of a UK beyond England). Towards the bottom of the piece was a mention of a mysterious poll that purportedly shows a "collapse" in support for Scottish independence - and yes, it turns out that I was far too generous to the Sun newspaper the other day, because at the time I was only aware of an article in which they had falsely claimed that the poll shows a "drop" in Yes support. Thanks to ConHome, I now realise there was another one that really did claim the poll shows a "dramatic collapse" for Yes.
The reality, as you'll recall, is that the Progress Scotland poll asked about independence in a completely new way, meaning there are no baseline figures to measure trends from. It's literally impossible to tell from the poll whether support for independence has gone up, down, or stayed static. Claiming a "dramatic collapse" on that basis is rather akin to saying that Neil Armstrong was much fatter than the previous man to have walked on the moon. But our plucky ConHome correspondent wasn't about to let troublesome things like facts get in his way - oh no, he was perfectly content to accept the Sun's outlandish claims as genuine, and gleefully seized upon the poll as proof that the "fragile Union" narrative is actually a "myth". You reach the stage where all you can do is point and laugh at these people. If the only comfort blanket they've got is a tabloid newspaper telling porkies about an opinion poll, they're really not in a good place. It's no more or less risible than Anas Sarwar brandishing his famous "top secret document" prop when all else failed.
Someone called "CMac11" left a comment on the previous blogpost and effortlessly solved the mystery of why the question format used by Progress Scotland appeared to produce lower support for independence than what we're used to from more conventional polling. He/she pointed out that no fewer than 17% of Yes voters in the poll rated themselves as ten on a scale of zero to ten, which supposedly indicated that they "completely support Scotland staying part of the UK". Self-evidently, that is an utterly implausible figure. In the whole five years since the independence referendum, I've met literally one single person who has converted from being a Yes voter in 2014 to being a committed Brit Nat now, and he only made that journey because of an idiosyncratic obsession with the named person scheme. Much more likely is that people who are no longer sure about indy would place themselves somewhere in the middle of the zero-to-ten scale. We can see how improbable a swing from one extreme to the other is from the fact that just 3% of No voters placed themselves at zero on the scale, meaning that they "completely support Scotland becoming independent".
Normally we can only speculate about the causes of seemingly illogical poll results, but this is an exception. It's blindingly obvious what's happened here. A significant minority of Yes voters clearly misread the question and thought they were indicating total support for independence (rather than total opposition) by rating themselves as ten on the scale. Why did only Yes voters make that mistake? Because it's natural to think that the higher end of the scale is intended to represent the maximum support for your own preferred position. No voters who made that assumption were correct, while Yes voters who made the same assumption were wrong. If the figures are adjusted on the reasonable assumption that approximately 14% of Yes voters mistakenly rated themselves as ten on the scale, that in itself would mean the poll underestimated support for independence by around 5% - enough to take us into roughly the same territory as the conventional polls.
So it's not just that it's impossible to compare this poll with previous independence polls and detect a trend. The poll itself is also an outright dud - albeit only in respect of the key question on support for indy, where something appears to have gone very badly wrong by complete accident. The next time you see a journalist or unionist politician point to this dud poll as proof that their "precious Union" is safe, do please try to keep a straight face.
I'm pretty sure there was an SNP canvassing card not so long ago which asked voters to rate their support for independence on a 0 to 10 scale, and 10 was wholehearted support for independence. People who has seen or used these cards would have been very likely to have gone for 10 if they were independence supporters.
ReplyDeleteWhat on earth was Progress Scotland thinking of?
It was a very very poorly worded question. Leads me to question the competence of Progress, much as I admire many of the folk involved in the organisation.
ReplyDeleteI totally agree this question was stupid. Expanding on past comments...
ReplyDeleteUsing a 0-10 'intensity' scale for how much you support two completely different (named as such) options is plain stupid.
It's like the Richter scale beginning with a catastrophic earthquake of ‘0’ intensity, increasing along the scale to a harmless, imperceptible tremor, before culminating in a city levelling day after tomorrow hurricane of force ‘10’.
Both ends must be 10/10 if you 'completely' support something. That or at least they should both come 'first place' 1/10, but certainly not 0 out of 10.
‘Aye, I’m totally zero out of 10 in support of that!’ has never in the history of humankind ever meant someone supported something ‘completely’.
As is stands, the question is being reversed half way through the scale, but the scale is staying the same.
Really, this could only have worked as (or similar for the supporting the union):
0 Don't support independence at all
10 Totally support independence
Then we have the problem of what is meant by 'completely'. I don't 'completely' support independence. I mean if it was being championed by the Scottish Nazi party who had cancelled democracy and were planning on a new tartan Reich I'd not be championing it; certainly not right now.
Then there's the date. Some people's support for independence is very date conditional (e.g. if brexit goes ahead...in a few years). What do they answer?
If they are completely against independence right now, but support it in a couple of months, how do they answer?
On top of this, the scale suggests all options are 'completely' as it has this at either end. If you are a 5, are you then ‘completely’ a 5? What of you feel you are not ‘completely’ any of these?
Respondents should never need to think about what the question is trying to ask, and therefore how they should answer it. If they're doing that, the results are already rubbish as people conclude on what is being asked in different ways. Compounding this is the natural quick unthinking ‘Im 10/10 for independence…next question please!’.
So, we have a stupid scale which, if reversed (10 for indy) would probably have resulted in overly high 'Yes' numbers.
At the same time, the question is not clear at all in what's being asked and dates involved, forcing the respondent to try and decide this for themselves.
All in all a total flop of a question and pretty much utterly useless.
I’m very surprised Survation let this one pass.
Your back are you.
ReplyDeleteThe union at Radio Scotland, have a similar approach to Scottish Independence.
ReplyDeleteTwisting a fact and, degrading anything Scottish Independence related, is the form expected at Radio Scotland.
Last time I heard that man J. Curtice on the station, he flung back made up [on-the-spot] figures back at the questioner, on the subject of Scottish Independence.
I did hear someone say, that the Scottish Progress poll questions, were defined by an experienced polling company.
Maybe Angus Robertson should think about retiring if this is going to be the level of his contribution from now on.
ReplyDeleteMaybe you should read up on the poll, then you'd know that:
Delete"Survey designed by Mark Diffley and data collected using the Survation panel in Scotland"
https://www.progressscotland.org/research/one-fifth-of-voters-are-open-minded-or-undecided-about-scottish-independence
Sounds like Mr. Diffley needs a kick in the bahookie for that question - definitely not up to scratch. As James said, the poll was a dud, and should be taken out of all calculations, which James will in his calculations, I am sure, because it is (a) not comparable; and (b) it is unsound from at least one point of view.
Delete"Survey designed by Mark Diffley and data collected using the Survation panel in Scotland"
DeleteSo PS are launching polls without the approval of their managing director?
You believe that e.g. the managing director of a company should personally undertake all the work of its subcontractors (e.g. designing specific survey questions), particularly if the latter are employed to provide 'expertise' that the company itself doesn't consider itself to have?
DeleteI do hope you are not in business.
Personally, I'd be looking for a partial refund over that question.
I'm happy to have Britnats brainwashing themselves the union is still safe and precious. No reason for them to be frightened about indyref2.
ReplyDeleteFrom my recollections of working with rating scales from a couple of decades ago 7 point scales were just as effective for measuring attitudes as 10 point scales. In addition they also resolve the problem of respondents misreading the end points of the scales, as seemingly happened here, as one can meaningfully name each point on the scale instead of asking for simple numerical ratings e.g. Strongly like through to Strongly dislike, Strongly approve to Strongly disaprove, etc as appropriate. if you really need to measure subtle differences one can even leave mid points between the named points on the scale but I bet factor analysis would demonstrate far fewer distinctions existed within the population being assessed than showed up on the scale. Using a 10 point scale that is so easily misinterpreted makes me doubt the competence of whoever constructed Progress' questionnaire.
ReplyDeleteI see the Tories are increasingly confident of working with Labour in a direct May/Mogg/Bozo/Corbyn Faragist Coalition.
ReplyDeleteDoes anyone know what effect such a coalition might have in Scotland? Do we have an example where Labour have teamed up with the Tories like this before?
Of course the coalition might not happen, but at least everyone knows Corbyn is absolutely 100% willing to go into one (otherwise there'd be no talks). This is important for the electorate.
Of course we don't need all of Labour to stand with the Tories to take Scotland out of the EU and single market. Just Corybn and a decent number of his nativist English nationalist mates.
I wonder what e.g. Ian Murray and his heavily pro-remain constituents think of this.
A less forgiving interpretation is people deliberately giving a false narrative of their previous vote in order to influence the outcome of a vote.
ReplyDeleteIn this case pretend you were Yes but now 100% No. If there's any demographic weighing going on that'd have a far higher impact on the result than an unchanging No. Same obviously goes for Yes.
Oh yes. We're all very human when it comes to ourselves.
DeleteI found myself thinking back to my grandfather telling me that " answering a stupid question is like talking to a lampost" that bit about trying to work out what the idiots meant was never meant to be argued with a shake of the head as a form of dismissal is the only answer to foolish questions. Don't give them credence by answering like some comments on some blogs no need to reply.
ReplyDeleteWhether it is a stupid question or not is really down to the attitude and opinion of the recipients. It may not necessarily be a stupid question��
DeleteRegarding the current formal coalition talks Corbyn is having with the Tories...
ReplyDeleteAnyone any idea what has happened to parties which have previously entered such a coalition? Wasn't there a party called the Lib Dems that did this? A google search suggests they used to get 20% of the vote in Scotland.
Mind that any 'agreement' has to be an ongoing coalition. The exit agreement and future relationship statement have to be adhered to. The former is legally binding, so they cannot fall out over it afterwards and start backtracking; that would be internationally ruinous. The UK would never get a trade deal again if just breaks what it signs up to immediately. As for the future relationship that; that is what the proposed coalition is about (currency union etc), i.e. Con and Lab delivering this together over the next 2 year transition period. So they must work together for the next few years to ensure that happens and the UK doesn't crash out of the EEA with 'no deal' at the end of it.
So this is not about Labour just supplying some votes one afternoon. Nope, it is about a grand nativisit Lab-Con English Nationalist coalition taking Scotland out of the EU and sending the furriners (and therefore businesses) packing.
Corbyn even said in the past "I might tell the jock scum 'now is not the time' if they ask for a referendum". He's really no different to the Tories.
Mum and I were on holiday in Venice and mum slipped on a fruit juice carton and almost feel into the water. I feel the Italians or the EU should have warned us this sort of thing could happen. It shows Nigel Farrage was right when he said all they want is our money and make us obey Sharon's law. I agree. Mum can't swim.
DeleteYou should have gone to The Alps, but maybe Mum would have slipped on a schnitzel wrapper and almost fallen off a mountain. Does she often slip in foreign beauty spots and almost do things?
DeleteAnd who's Sharon? Did she almost do something too?
DeleteFirst Class analysis which explains this curious poll. However you ask the reasonable question is there anything more ridiculous than Tory commentator misreading an anti-independence opinion poll. Yes I am afraid there is. And that is a private research company (one endorsed by the SNP leadership) taking hard earned money off hard working Indy supporters and then promoting a poll with negative results right up on its website, then taking down the detailed table which maximised the anti Indy publicity in the MSM, and now being exposed as simply incompetent by this excellent website - all of this with no apology, explanation or willingness to improve. That is not “progress”.
ReplyDeleteI knew people called Mackintosh from Brechin. Are you related? Did they stay in New Zealand?
Delete