A wander through the YouGov and Panelbase datasets

I've now had a look at the datasets from both the YouGov and Panelbase polls - the Panelbase datasets aren't on the firm's website yet, but Ivor Knox very kindly sent me a copy.  Of course the first thing I always look for is the unrounded voting intention figures (or rather the figures rounded to only one decimal place), and on that front there is good news and bad news...

YouGov :

Yes 51.2% (+4.4)
No 48.8% (-4.4)

That's better than I originally expected, and the reason is that the YouGov poll was actually slightly misreported in many media outlets - the split with Don't Knows included is Yes 47% (+5), No 45% (-3), rather than Yes 47%, No 46%.  As a result, I've had to make a slight correction to the latest Poll of Polls update - the average No lead is now a further 0.2% smaller.

The less good news is that Yes only barely made it to 48% in the Panelbase figures, thanks to the effect of rounding -

Panelbase :

Yes 47.5% (-0.1)
No 52.5% (+0.1)

So that further deepens the mystery of why the Panelbase numbers don't appear to be budging at all (albeit from an all-time high position for Yes), at exactly the same time as the No vote appears to be in freefall with the formerly No-friendly firm YouGov.  The gender gap has been more or less wiped out in the Panelbase poll - No lead by 53.2% to 46.8% among women and by 51.7% to 48.3% among men, which reverses a modest Yes lead among men in the last poll.  It would be tempting to believe that the swing to Yes among women must be real and that the swing to No among men is an illusion, but there's a danger of wishful thinking in that kind of speculation - it's just as possible that normal sampling variation is at play, and that the Yes figure is a bit too high among women and a bit too low among men.

Nevertheless, it's true that if you isolate out the figures for women only, today's two polls are actually identical, because YouGov are also showing figures of Yes 47%, No 53% for female respondents.  Moreover, YouGov concur that there has been a narrowing of the gender gap - the swing to Yes among women since Tuesday's poll is a full 5%, while among men it's only 3%.  But even though two polls are suggesting that the female Yes vote is closing the differential, that is still far from being absolute proof that it's really happening.

One thing that is really striking about a comparison of the two polls is that on the raw unweighted numbers, the Yes vote remains higher with Panelbase than with YouGov - but Panelbase's weighting procedures have harmed Yes, while YouGov's weighting procedures have harmed No.  Of course to some extent that is for very good reasons - YouGov's raw sample significantly under-represents lower income voters, who are more likely to be in the Yes column.  But if I was going to raise a question mark, it would once again be over Panelbase's recent introduction of weighting by recalled European election vote, which no other firm is doing and which significantly harms Yes - the 278 people who recall voting SNP in May have been downweighted to count as just 221 people.

By the same token, eyebrows might be raised at the fact that YouGov's weighting by Holyrood vote from 2011 has helped Yes - the 331 SNP voters in the raw sample have been upweighted to count as 375 people, while Labour voters have been significantly downweighted.  But the crucial difference is that YouGov know for a fact that they have an in-built problem in their panel with having too many Labour voters from 2011, because they collected detailed information at the time, and a large chunk of their current panel were already with them back then.

Panelbase and YouGov now have something important in common - they're the only two pollsters so far who have taken the sensible step of introducing weighting by country of birth, which we know is a strong predictor of referendum vote.  Both pollsters are now showing a clear lead for Yes among Scottish-born respondents, although as you'd expect the lead is slightly bigger with YouGov (54% to 46%, compared to Panelbase's 52% to 48%).  And of course in both cases, those respondents have had to be upweighted while English-born respondents have had to be downweighted.  If all six pollsters were doing this, it's reasonable to imagine that the average No lead on the Poll of Polls would be a touch lower.

Paradoxically, though, what's really setting the YouGov poll apart from Panelbase is the findings among respondents who weren't born in Scotland - YouGov say that 32% of people from other parts of the UK, and 46% of people from outside the UK, are now planning to vote Yes.  Panelbase don't have exactly equivalent figures, but they're saying that only 26% of English-born respondents and 24% of respondents born outside both Scotland and England are Yes voters.  If those figures are underestimates (and for what little it's worth, my gut feeling is that they are), that could partly explain why Panelbase are failing to detect the swing reported by YouGov.

Indeed, what's really encouraging about the YouGov poll is that, almost right across the board, demographic groups that have hitherto resisted the Yes message are swinging in our direction.  It's not just women and people born outside Scotland - the Yes vote among over-60s has increased from 31% in Tuesday's poll to 38% today.

Irritatingly, YouGov have departed from their previous practice of showing turnout-filtered voting intention figures in their datasets.  However, the likelihood to vote looks almost identical among Yes voters and No voters, so it seems reasonable to suppose that Yes have exactly the same 51% to 49% lead among definite voters (which would mean that the swing to Yes since Tuesday's poll is 5% among definite voters, 1% higher than among the whole sample).  By contrast, Panelbase's turnout filter is helping Yes - without it the No lead with DKs excluded would be 2% higher.

Final thought : fresh from his red herring yesterday about "expectation management", Kenny Farquharson is now trying to set another hare running about how Yes supposedly need more than 51% in the polls to have a chance of winning, because of the large number of postal votes that were cast when No were still clearly in the lead. Not to put too fine a point on it, that's utter garbage.  If you've already voted, and a pollster asks you how you intend to vote, you're highly likely to tell them how you actually voted.  That means that it's a touch harder for either side to achieve a swing in the polls, but that any Yes lead still means exactly what it says.

When I pointed that out, Kenny said : "That assumes people with postal votes don't change their mind."  Well, no it doesn't actually, but even if we assume for the sake of argument that some people are telling pollsters they plan to vote Yes when they have in fact already voted No (!), that is nowhere near the disadvantage for Yes that Kenny seems to think it is.  Nobody voted by post until after the second debate, and the first two post-debate polls had Yes at 47%.  If the postal votes reflect that position, and we assume that they account for roughly 1 in 5 of all votes cast, then Yes would only need to be on 51% of the vote by polling day to win. (Do the sums yourself if you don't believe me!)