Friday, February 18, 2022

Why the results of the BBC / ComRes poll on GRA reform are less reliable than the three recent polls showing massive opposition to gender self-ID

You may have seen yesterday that BBC Scotland commissioned a poll about reform of the Gender Recognition Act.  Having been through the process of commissioning a GRA poll myself a few months ago, and having approached more than one polling firm before ending up with the excellent Panelbase, I'm instinctively very sceptical about any poll that purports to show strong support for the proposed reform, because I gained the firm impression that some pollsters themselves live in fear of being branded 'transphobic', and that question wordings can end up being heavily skewed by that fear.  Even if BBC Scotland themselves approached the exercise without an agenda, it's entirely possible that the intended questionnaire may have 'evolved' quite radically after engagement with the polling company.  As soon as I looked at the Savanta ComRes data tables, alarm bells started ringing in my head.  Just to take one trivial example, it's stated that there was one question that was asked only to "cisgender" respondents.  Given that the use of the word "cisgender" is strongly opposed by one side of the debate (indeed many gender critical feminists find the word highly offensive), that does not inspire confidence that the devisers of the poll were coming from a place of studied neutrality.

And I would also have to say that some of the mainstream media's reporting of the poll yesterday was truly dire.  In particular I would single out The Scotsman's reporting, which seemed to be going out of its way to give the misleading impression that there is widespread public support for the contentious proposed reforms.  This is becoming an established pattern for The Scotsman, and it's really quite curious for a publication that has a long-earned reputation for being conservative with a small 'c'.  My guess is that a small group of young reporters have essentially seized control of the paper's editorial stance on the GRA and have turned it into an unlikely bastion of identity politics zealotry.  Essentially the con was that they drew attention to the wrong question and portrayed it as if it was the 'headline result' that measured public opinion on the matter of controversy, when in fact it did no such thing.  Here is the exact wording of that question -

Given this information on the previous pages, to what extent, if at all, would you support or oppose making the process to acquire a Gender Recognition Certificate easier for transgender individuals?

First of all, that's a 'motherhood and apple pie' question which is deliberately framed to make it as difficult as possible for respondents to give a negative response - few people will want to disagree with the idea that life should be made easier for a minority group.  But even leaving aside the issue of virtue signalling, many opponents of gender self-ID either actively support making the process of obtaining a certificate easier, or are genuinely open to the idea.  So to portray the substantial majority on this question as proof that the public are overwhelmingly behind the Scottish Government's specific proposed reform is an extremely cynical red herring.  I'm also puzzled by the words "given this information on the previous pages", because there doesn't actually appear to have been much information on the previous pages. All there seems to have been is a brief definition of the word "transgender" (unless something is missing from the data tables).

So the really pivotal question is not the fluffy one about whether making life easier for people is a nice idea - it's the specific one about gender self-ID.  And that produced a much closer result.

To what extent, if at all, do you support the following amendments to the process for obtaining a Gender Recognition Certificate in Scotland?

Allowing transgender people to self-identify by removing the requirement for a medical diagnosis of gender dysphoria (a condition where a person experiences discomfort or distress because there's a mismatch between their biological sex and gender identity)



That's a statistical tie, and to the extent that the media gave this result the coverage it should have warranted, the failure of journalists was to not put it in the context of the three preceding polls in recent months that showed a very different picture.  The Scot Goes Pop / Panelbase poll, the Murray Blackburn Mackenzie / Survation poll, and the For Women Scotland / Panelbase poll all reported massive opposition to self-ID.  It's unlikely that public opinion has been transformed so rapidly in such a short space of time, so the reason for the contradiction is likely to be bound up in methodology or the question wording.  I suspect it's primarily to do with a lack of specificity in the ComRes question, which may mean that many respondents weren't clear about what they were being asked.  

In fairness, the question is nowhere near as dreadful as the notorious Pink News / YouGov poll that asked an ultra-short and ultra-vague question about whether people should be able to self-identify, without even a shred of explanation of what the concept of self-ID actually means (many respondents may have thought they were being asked about a "state of mind", rather than a concrete change in legal status).  ComRes do explain that self-ID discards the need for medical diagnosis, and they do explain that the question is being asked in relation to the process of obtaining a Gender Recognition Certificate. But what appears to be totally absent is any explanation of what a Gender Recognition Certificate is, or the rights that are gained by obtaining one.  By contrast, the self-ID question in the Scot Goes Pop / Panelbase poll was crystal-clear that what was being asked about was a legal change of gender or sex that would be written onto an individual's birth certificate.  

On an issue like this where the level of public awareness is relatively low, the most accurate or meaningful results will be gained from poll questions that are the most specific and informative, and the least meaningful results will come from poll questions that are the most vague.  The ComRes question is somewhere on the middle of that continuum, and I would therefore suggest that the results are somewhat less credible than the three recent polls showing wide-scale opposition to self-ID.

Nevertheless, the ComRes poll does pick up public opposition to specific parts of the proposed reform, namely that the time people must live in their acquired gender before obtaining a certificate should be reduced from two years to six months, and that the minimum age at which a certificate can be obtained should be reduced from 18 to 16.

*  *  *

Please bear with me as I continue promoting Scot Goes Pop's fundraising drive.  Opinion polls are so expensive that since I started commissioning them, fundraising has almost become like painting the Forth Bridge.  If you'd like to help this blog continue for another year, or to help us commission another full-scale poll like the six we've commissioned over the last two years, here are the various options for donating...

Via the Scot Goes Pop polling fundraiser for 2021-22, which I set up in the autumn and is part-funded.

If you prefer to donate directly, that can be done via Paypal or bank transfer:  

My Paypal email address is:

Or email me for my bank details.  (My contact email address is different from my Paypal address, and can be found in the sidebar of the desktop version of the site, or on my Twitter profile.)

1 comment:

  1. Agree completely with your analysis James.
    This survey was conducted by a reputable polling company but the way they ask the questions, the order they ask them and even the explanatory text can have a dramatic effect on results.

    Stonewall commissioned a poll some time ago and use selected headline figures in their campaigns on gender reform. If you look at the more detailed figures as a whole, though, a different story emerges.

    Like this one, If I remember correctly, one of the questions was about transgender women having access to women's spaces like toilets and changing rooms. The response was, generally, to accept them. When the question was reframed to include transgender women who had not had reconstruction surgery - ie with full male genitalia - the response was much more negative. Stonewall do not highlight this in their campaigning.

    The problem with surveys of this type is that most people do not follow the debate closely and, therefore, do not always appreciate the implications of the proposals. With many people having the desire to "be nice" as we all know, even a subtle change to the way questions are asked can skew the response.
    Your poll and the two others you mention are pretty much in accord. As is the Irish poll for The Countess' carried out by RedC. The most interesting fact being this was carried out 5 years after Self ID was snuck in along with same sex marriage in the Irish Republic. Would seem the public aren't too enamoured by (what we're told is) 'International Best Practice'.
    BTW did you notice the questions on allowing primary school children to change gender and access to trans literature/materials in school.
    Parents voted overwhelming against. I wonder how many parents realise the Scottish Government has already issued guidance to schools to allow this very thing.
    Doubt we'll be seeing that reported.