tag:blogger.com,1999:blog-930120922627919768.post7527406605504383730..comments2024-03-18T22:55:22.060+00:00Comments on SCOT goes POP!: To preamble or not to preamble - is that the question?James Kellyhttp://www.blogger.com/profile/01516007141763230886noreply@blogger.comBlogger3125tag:blogger.com,1999:blog-930120922627919768.post-76279320109382504002014-02-12T12:46:46.485+00:002014-02-12T12:46:46.485+00:00"At the moment if we're implying that thi...<i>"At the moment if we're implying that this outlier is more likely to be correct than every other poll on the subject, then we're also assuming that the question order has a negligible effect."</i><br /><br />But I am categorically NOT implying that, and I'm struggling to understand how you ever formed the impression that I am. To reiterate yet again what I am actually saying, we HAVE NO IDEA whether the different result was caused by the neutral preamble, or by the question sequence. To repeat what I said in my very first reply to you, it may well be a combination of both factors.<br /><br /><i>"doesn't get us to the conclusion you're trying to draw here: that this poll is correct and every other one is wrong."</i><br /><br />Again, that is NOT the conclusion I'm trying to draw, and I'm baffled as to why you think it is.<br /><br /><i>"If you put on a blindfold and throw 100 darts at a list of percentages then you might hit the right one, but there's no way to know which of the 100 darts is correct."</i><br /><br />Which is what I was getting at earlier - that is the case that can be made for thinking that an average is likely to be <i>less inaccurate</i> that any other method that can be devised. But there's no reason to suppose that it will turn out to be particularly accurate, or that it will be closer to the truth than any given outlier.<br /><br />Incidentally, there IS a way to form a rational view on which polls are more likely to be accurate - and that is to consider the merits of their methodologies. Which is exactly what we're trying to do.James Kellyhttps://www.blogger.com/profile/01516007141763230886noreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-74574653686310014232014-02-12T11:53:02.107+00:002014-02-12T11:53:02.107+00:00First I should say I'm not the famous Bill Wal...First I should say I'm not the famous Bill Walker, just a regular Bill (unfortunately/fortunately). <br /><br />Let's be clear, though, I'm not saying that the preamble has no effect - nobody really knows what effect it has on the results. However you can say the exact same thing about the ordering of the questions. If anything that seems a far more likely case of something that could skew the results given multiple surveys in the past have shown that those two questions promote very large pro-Scottish responses (Scotland can be an independent country, and the Scottish government is more trustworthy than Westminster).<br /><br />At the moment if we're implying that this outlier is more likely to be correct than every other poll on the subject, then we're also assuming that the question order has a negligible effect. In other words we're privileging one type of bias over another - given no other polls that I'm aware of have either used this preamble or the same ordering of questions.<br /><br />Polling methodologies are flawed and we can come up with countless examples through the years of them getting it wrong. However that in itself doesn't give any reason to think a particular outlier is accurate. If you put on a blindfold and throw 100 darts at a list of percentages then you might hit the right one, but there's no way to know which of the 100 darts is correct.<br /><br />While we can offer some qualifiers about never really knowing whether an opinion poll is accurate or not, that doesn't get us to the conclusion you're trying to draw here: that this poll is correct and every other one is wrong. <br /><br />A far better route if you want to look for some optimism is to look at the trends of polls using the same methodology - given you can eliminate most of the problems with bias (although you also have to be careful that the changes are beyond the margin of statistical error, which is about 3% in most polls).Bill Walkernoreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-61444178710337022092014-02-12T08:16:17.629+00:002014-02-12T08:16:17.629+00:00If we had a spread of bias in the polls, an averag...If we had a spread of bias in the polls, an average might work, but as the vast majority are biased to pro-union, an average does not work.Anonymousnoreply@blogger.com