Pro-independence campaign remain on all-time high of 47% in new Survation poll

A new Survation poll is out tonight, and it has more or less crushed the fantasy that there has been any movement back to No of late.  The Yes campaign remains on the all-time high of 47% that it first reached two months ago -

Should Scotland be an independent country?

Yes 47% (n/c)
No 53% (n/c)

When Don't Knows are taken into account, the figures are -

Yes 40% (-1)
No 46% (n/c)

However, even that minor percentage change is misleading, because on the unrounded numbers Yes and No are both down by a trivial amount - Yes by 0.7%, and No by 0.1%.  In other words, there is no change of any statistical significance whatsoever, and the fact that the No campaign have rushed out their customary glossy graphic about "No Thanks extending lead" may well be what the word "desperate" was invented for.

Unlike last weekend's Panelbase poll, Survation have made no methodological changes (none have been announced at any rate), and therefore tonight's figures are directly comparable with previous Survation polls.  However, for my money this poll may actually be slightly more encouraging for the Yes campaign than the previous two polls from the same firm, even though on the face of it the results are identical.  When I saw the datasets for those last two polls, my heart immediately sank, because there was a very simple way in which it could be argued that the heady heights that Yes had reached were illusory.  In both cases, the most Yes-friendly age group was 16-24 year olds, and they had been upweighted to an extreme degree, meaning that any error in the voting intention figures for that group caused by random sampling variation would have been magnified in the overall results.  But in tonight's poll, although 16-24 year olds have again been dramatically upweighted from 45 real respondents to 130 'virtual' respondents, that doesn't matter as much, because they're now only the fourth most Yes-friendly age group - or, to put it another way, the third most No-friendly age group.  So this is in fact the first ever Survation poll to put Yes as high as 47% without there being anything weird going on due to age-related weightings.

Unfortunately, there is still one caveat - as Alistair Davidson points out in the comments section below, respondents from the South of Scotland have been upweighted two-fold, and that means what is almost certainly a highly inaccurate breakdown of Yes 50%, No 39% for the region has been magnified.  However, that problem will have been at least partly offset by the more modest upweighting of a dubiously big No lead in Mid-Scotland and Fife.  The best evidence that Yes have not been artificially flattered in this poll can be found in the raw unweighted numbers, which are very similar to the published results.  That was not the case last month - indeed in last month's poll there was a sharp fall in the unweighted Yes vote, which led me to have a nagging doubt at the back at my mind as to whether Survation's weighting procedure was masking real slippage for Yes.  Thankfully, we can now relax on that score.

Survation's own commentary on the poll suggests that there has been no Commonwealth Games boost for Yes - which is absurdly premature.  Come to think of it, a few of you might recall some chap at the Spectator getting terribly excited by the first post-White Paper poll failing to show much movement, when in retrospect it's blindingly clear that the White Paper produced a significant pro-Yes swing.  As I explained above, it's possible that in an underlying sense this new poll is better for Yes than the two previous Survation polls, which if true could mean that polls from other companies will show a swing on their headline numbers.  But it's also possible that the impact of the Games will not be felt immediately - the referendum campaign has been put on pause to some extent over the last ten days, and it will only be when it restarts that we'll see whether the greater confidence generated by Glasgow's time in the sun will lead people to be more receptive to the case for independence.  And the final point to make is that Survation's own supplementary question on the issue contradicts their own conclusion to some extent - 12% of undecided voters say the Games have made them more likely to vote Yes, compared to 4% of undecideds who are more likely to vote No.  True, that's only a small advantage for Yes - but then No are only defending a very small lead.

There's also a question asking who people expect to win the Salmond v Darling debate, with 37% plumping for the First Minister, and only 11% for David Cameron's stand-in.  Survation claim that this "falls short of a ringing endorsement" for Salmond - which could be an extremely good thing.  "Winning" a debate is to a large extent a question of exceeding expectations, and if as many as 63% of the population don't realise that Salmond is a better debater than Darling, that'll make it a hell of a lot easier to exceed expectations.

* * *

REQUIRED SWINGS

Swing required for 1 out of 6 pollsters to show Yes in the lead or level : 3.0%

Swing required for 2 out of 6 pollsters to show Yes in the lead or level : 3.5%

Swing required for 3 out of 6 pollsters to show Yes in the lead or level : 4.5%

Swing required for 4 out of 6 pollsters to show Yes in the lead or level : 5.5%

* * *

SCOT GOES POP POLL OF POLLS

I didn't get round to posting an update of the Poll of Polls after last weekend's Panelbase poll, so this update takes account of both Panelbase and Survation. The slight increase in the No lead should therefore be taken with a heavy dose of salt, because it may well be caused by Panelbase's big methodological changes. Ironically, of course, the Survation and Panelbase results will actually increase the Yes vote in other Polls of Polls, which mostly use variants on the Curtice method.

MEAN AVERAGE (excluding Don't Knows) :

Yes 43.1% (-0.5)
No 56.9% (+0.5)

MEAN AVERAGE (not excluding Don't Knows) :

Yes 36.3% (-0.5)
No 48.0% (+0.3)

MEDIAN AVERAGE (excluding Don't Knows) :

Yes 43.4% (n/c)
No 56.6% (n/c)

(The Poll of Polls is based on a rolling average of the most recent poll from each of the pollsters that have been active in the referendum campaign since September 2013, and that adhere to British Polling Council rules. At present, there are six - YouGov, TNS-BMRB, Survation, Panelbase, Ipsos-Mori and ICM. Whenever a new poll is published, it replaces the last poll from the same company in the sample. Changes in the Poll of Polls are generally glacial in nature due to the fact that only a small portion of the sample is updated each time.)

Yet another Ipsos-Mori megapoll underway, possibly paid for by the taxpayer - will this one be published?

From the sheer number of people that have reported being interviewed over the last few days, it's obvious that Ipsos-Mori are conducting a referendum poll on an absolutely mind-boggling scale - it must be a sample size of at least 10,000, which will be costing the client an absolute fortune.  We know that client is on the No side, because the questions are skewed in that way.  It could be the official No campaign, but on past form it's much more likely to be the UK government paying for a poll with taxpayers' money - and then sharing it with the No campaign for free.  In other words, we are all subsidising the No campaign's private polling, and we won't even see the results unless they happen to be particularly favourable for No.

Let's make sure we don't stay quiet about this bloody outrage, shall we?

Commonwealth Games photos

I won't try your patience with too many of these, because I know the picture quality is somewhat less than brilliant.  But in reverse chronological order, here's a brief overview of what I've been seeing in Glasgow over the last week-and-a-half.  I bought the tickets ages ago (long before I ran the fundraiser, I hasten to add!), and with one exception I chose the cheapest option for each sport, so I could see the widest variety possible.  The problem with taking that approach is that you end up seeing a lot of fairly routine preliminary-round matches.  On the first day when the heatwave was at its peak, I sat in unbearable heat watching South Africa beat Trinidad and Tobago 16-0 in a tedious women's hockey match from an absolutely terrible vantage point, and I thought "this could start to get wearing".  But I had much more luck with sessions in other sports, and to my surprise I saw quite a few quarter-finals and semi-finals.  I think the highlight may well have been tonight at the squash, when I saw Scotland's Alan Clyne and Harry Leitch storm into the semi-finals of the men's doubles in an electric atmosphere at Scotstoun.  I think I'm right in saying that Scotland haven't won a squash medal of any colour since Peter Nicol took gold in Kuala Lumpur in 1998, although of course irritatingly we had to watch as Nicol claimed several more medals for England in 2002 and 2006 after defecting.  So Clyne and Leitch are potentially only one game away from ending the drought.

The band you can see in one of the photos is the Boomerang Project - a collaboration between the amazing Scottish band Breabach, and Maori and Australian Aboriginal musicians (including a former Australian Idol winner).  It was put together specifically for Commonwealth Games year, and the common theme is the suppression of language and culture - Gaelic, Maori and Aboriginal.  This is a strikingly different vision of what the Commonwealth is all about - more a victim support group, rather than Lord Coe's notion of former colonies genuflecting towards their vestigial Britishness.

Wisdom on Wednesday : The housemates up for eviction this year are...

"A Yes for independence, Michelle Mone leaves. A No for independence, I leave. Vote wan of us oot! What's it to be?"

'Still Game' star Greg Hemphill, presenting the choice we're faced with this September in its starkest form yet. I'll need to take 0.000000000000000000000000001 microseconds to think about this one.

(And please don't feel guilty. When the inevitable happens, an excitable host will be on hand to reassuringly whisper in Michelle's ear "It's just pantomime booing, babes".)

A Commonwealth Games FAQ, brought to you by George Orwell

Q.  Can I bring the flag of a political cause or of a non-competing nation to the venue?

A.  Absolutely NOT.  The flags of political causes and of non-competing nations are strictly forbidden, and if you wave one at the venue you will be removed by the police.

Q.  The UK is a non-competing nation.  Is it OK if I wave a Union Jack?

A.  Yes, of course it is!  But please remember that waving the flags of non-competing nations is not allowed.

Q.  Can I consume any flavour of ice cream at the venue?

A.  Regrettably not.  Due to sponsorship arrangements, only the consumption of raspberry ice cream will be permitted.

Q.  Vanilla is David Cameron's favourite flavour.  Is it OK if I consume vanilla ice cream at the venue, even though only raspberry is permitted?

A.  Yes, of course you can!  But please remember that only the consumption of raspberry is permitted, and if you try having strawberry or any other non-raspberry flavour that isn't David Cameron's favourite, the police are likely to intervene.

Q.  Is it OK for contributors to the BBC's coverage of the Games to make comments that might be construed, however implausibly, as favouring one side or the other in the Scottish independence referendum?

A.  Categorically NOT.  Surely you saw that Chris Hoy couldn't even use the word 'Yes' when he was talking about a marriage proposal, and instead had to weirdly use the word 'acceptance'.

Q.  But even though it's not OK for the BBC coverage to be seen to take sides in the referendum, is it still OK for BBC hosts such as Gary Lineker to presuppose a No vote by endlessly inviting people like Ian Thorpe and Chris Hoy to speculate on the prospects for "Team GB" in Rio, without ever bothering to issue a disclaimer that there probably won't be a Team GB in Rio if Scotland votes Yes?

A.  Did you really need to ask?

The Gaelic musical that likes to say Yes

Ah, the Commonwealth Games.  Is this the greatest show that the far north of "the Middleland" has ever seen?  I've been so busy milling around Glasgow over the last few days that I haven't really had a chance to catch up with the reaction from other countries, but I was thrilled to spot a rave review on the CBC (Canadian TV) website last night.  Of course there have been lots of cultural events running in parallel to the sport, and last night I headed to Glasgow Green to see a free performance of Children of the Smoke, a sort-of-musical (although that doesn't quite do it justice) in Gaelic.  Unfortunately I missed the start, which probably means that I also missed a rough synopsis, so I couldn't make much sense of what was going on - but it did seem pretty extraordinary.  I spotted Kathleen MacInnes in the cast (although I must admit it took me 20 minutes to work out where I recognised her from!), and from what I read afterwards Patsy Reid may have been one of the live musicians, but as you can see from the photos below I was in the wrong vantage-point to see the musicians on the left.

In the last photo, you can probably spot a couple of cards on the side of the boat.  If the camera on my phone wasn't quite so rubbish, you'd also be able to spot that they display the Yes logo!  They were left up for a good couple of minutes at the end of the show.

Here's why the No campaign should be worried about the British Election Study data

Over the last day or two, you might have seen the referendum data from May/June published by the British Election Study. It shows a relatively narrow (albeit hardly untypical) No lead of 51% to 39%, which is down from a 52% to 37% split in February/March.  But what I didn't realise initially was that the fieldwork was conducted by YouGov, whose panel is notorious for producing higher No leads than the average (both in polls for YouGov themselves, and in Progressive Partnership polls for which YouGov conducted the fieldwork). There has been no official YouGov poll so far in this campaign that has shown a No lead of as low as 12% (the lowest was 14%), and nor has there been one that has shown a Yes vote of as high as 39% (the highest was 37%).  Yet the unusually narrow gap in the BES data has been found in spite of the sample size being five times bigger than for a standard YouGov poll.

Overnight, I was sent the following commentary on the survey results by someone who said "no need to mention me" - I'm not sure whether that was a specific request for anonymity or not, but to be on the safe side I'll assume that it was!  Incidentally, I've changed the Wave 1 Yes figure below from 38.6% to 36.6%, which I'm pretty sure is the correct number.

I am a YES supporter and canvass in Edinburgh. I am not an expert on opinion poll methodology but analyse statistical data from scientific experiments so have some familiarity with issues involved in estimation.

I thought I would pass on some information about the "British Election Study" that was released today. I was at the academic launch meeting in Edinburgh today.

Some detail on methodology

The data was collected **online** by **YouGov** on behalf of BES. The Wave 1 (i.e. 20 Feb to 09 March) sample was N=5896 and Wave 2 (i.e. 22 May to 18 June) was N=6182. The percentages for Wave 1 was 52.0% NO, 36.6% YES and 11.4% undecided. Wave 2 was 51.0% NO, 38.8% YES and undecideds 10.2%. I copied these figures from the powerpoint presentation of Prof Ed Fieldhouse.

Although it can be classified, with regards the IndyRef, as a YouGov poll, many other questions were added (over 200, on a wide range of topics). These 6000 Scots voters were part of a larger sample that had 20,000 English and 3,000 Welsh. The Scottish referendum question was not asked upfront therefore. Also, as part of a GB poll, it did not include voters who were 16 and 17.

Prof Curtice chaired the meeting and commented that, for a YouGov poll, it was not as NO-friendly as expected.

I have some other hand-written notes but I hope this helps to put it in context.

NOTE for OPINION POLL GEEKS (like me :-)

The BES polling data was collected by YOUGOV on behalf of BES.

This poll should thus be compared to recent YouGov IndyRef polls. YouGov is one of the 6 polling companies charting the progress of the referendum, and tends to produce high leads for the NO camp. Polls have shown NO leads ranging from 3% (Panelbase) to 19% (YouGov) recently.

These results above show NO at 51.0%,YES at 38.8% and UNDECIDED at 10.2% (data collected 22 May to 18 June, sample size 6162) at the later date and NO 52.0%, YES 36.6%, UNDECIDED 11.4% (data collected 20 Feb to 09 March), sample size N-5896) at the earlier date.

Note also that 16 and 17 year-olds were not included.

COMPARISON with YouGov polls

These N=6000 polls can be compared with YouGov polls with N=1200 samples. However, note that these larger polls ask other questions (over 200 questions) and the Scottish IndyRef question is not asked upfront. Note also that 16-17 year olds were not included in the N=6000 polls.

N=6000 polls and N-1250 YouGov polls

These two snapshots in Feb/March and May/June have NO leads of 15.4% and 12.2% respectively, based on samples of 6000 voters each.

Standard Indyref polls with sample sizes of about 1200 in approximately the same period showed a NO lead of 18% (24-28 March, N=1257) and a NO lead of 19% (25-29 June, N=1206).

* * *

One point I should add for clarity is that YouGov never interview 16 and 17 year olds, even for dedicated referendum polls - they're the only pollster that still fails to do so.

Panelbase announce huge methodological changes - the new poll is not directly comparable with anything that has gone before

Well, this is a bit startling.  It turns out that the exchange/argument/debate a number of us had on the previous thread was based on an entirely false premise, and indeed on a false premise twice over.  It was certainly the case that in their last poll for the Sunday Times back in May, Panelbase continued to use a slightly different question from the one they use for their other clients - although the full question didn't appear in the datasets, we know it was different because Calum Findlay took part in the poll and quoted it in full.  But if the datasets for the new Sunday Times poll are to be believed, the question has finally been brought into line with other Panelbase polls.  So in that sense the poll is directly comparable to the last Panelbase poll for Yes Scotland - but that fact has been totally eclipsed by the revelation that two other huge methodological changes have been made, meaning that no direct comparison with any previous Panelbase poll is possible.

The first change is one that I find extremely troubling, because it could be interpreted as the first sign that Peter Kellner's attempts to browbeat his fellow pollsters into adopting a more No-friendly methodology have borne fruit.  Basically Panelbase have decided to weight their results by a mixture of how people recall voting in the European elections in May, and how they recall voting in the Holyrood election in 2011.  Because too many people in the Panelbase sample recall voting SNP in May, the effect of adding a European weighting is to increase the No lead.  The problem with this approach was covered in Survation's response to Kellner's notorious article - they pointed out that far too many people recall voting in the European elections, full stop. So either the samples in online polls are hopelessly unrepresentative of the general population (in which case no amount of weighting can be sure of correcting the problem), or else a large number of people are saying they voted in May when they didn't.  If by any chance the latter is the case, downweighting respondents who claim to have voted SNP in May could be a monumental error, because it's perfectly possible that people who falsely say they voted are disproportionately likely to also say that they voted SNP.

I can only hope that no other pollsters follow suit.  This change would only be justified if respondents' recall of whether or not they voted in May bore at least some resemblance to the actual turnout.  That is clearly not even close to being the case.

On a more positive note, the second change is one that is long-overdue across the entire polling industry - Panelbase have started weighting by country of birth.  For some reason most pollsters seem to end up with too many English-born respondents in their sample, which probably goes some way towards explaining the disconnect between canvassing returns and published polls (although it certainly can't explain all of the disparity).  So in the new poll English-born respondents have been heavily downweighted from 162 to 94, while Scottish-born respondents have been upweighted from 794 to 864.  It's possible that Panelbase are the first company to introduce this form of weighting - the only one I'm not sure about is Ipsos-Mori, who routinely ask for people's country of birth as a "demographic" question (which implies that they weight by it), and yet they still seem to end up with too many English-born people in their final results.

Panelbase imply that the two changes have effectively cancelled each other out, although I'm slightly sceptical about that.  The precise wording used is "the net effect of these two new weights is statistically insignificant", and yet it's later suggested that changes between other recent Panelbase polls have also been statistically insignificant. So it seems this definition of statistical insignificance can encompass changes of 1% or greater - which in turn implies that it's perfectly possible the No lead would not be 7% in the new poll if the old methodology was still being used. So the claims of the No campaign that their lead has increased by 4% since the last Panelbase poll should be taken with a lorry-load of salt.

I'm very grateful to Ivor Knox of Panelbase for copying me in on an email that was sent to all of the firm's recent referendum clients (presumably the Sunday Times, Yes Scotland, the SNP, Wings Over Scotland and Newsnet Scotland). The main aim of the email seems to be to head off any "conspiracy theory" that Yes would have taken the lead in the new poll if the methodology hadn't been changed - "the old weighting would also show a No lead" is the emphatic message. However, there's no additional information about whether that lead would have been as big as 7%, which takes us back to the point I made above.

What is revealed, however, is that if weighting by past vote recall had only factored in the European elections (as opposed to both the European and Holyrood elections), and if there had been no weighting by country of birth, the Yes vote after Don't Knows are excluded would have been in the region of 42-43%. That's fascinating, because it more or less explains the difference between YouGov and the other online pollsters - if Panelbase make the sort of adjustment to their past vote weighting that Peter Kellner would approve of, it's enough to take their numbers down to YouGov-type levels, whereas by the same token it's presumably the case that all YouGov would have to do to produce Panelbase/Survation-type numbers is ditch the artificial "Kellner Correction" and introduce weighting by country of birth.

Another important question is what would have happened if the only methodological change Panelbase had made was the introduction of weighting by country of birth. It clearly wouldn't have been quite enough to push Yes above 50%, but the gap would certainly have been narrower.