tag:blogger.com,1999:blog-930120922627919768.post5378570021047833970..comments2024-03-29T07:48:22.768+00:00Comments on SCOT goes POP!: Concerns raised over Ipsos-Mori's methodologyJames Kellyhttp://www.blogger.com/profile/01516007141763230886noreply@blogger.comBlogger22125tag:blogger.com,1999:blog-930120922627919768.post-77882164817391469562014-04-04T19:24:00.759+01:002014-04-04T19:24:00.759+01:00Momentum?Momentum?Rolfehttps://www.blogger.com/profile/17849975010197698907noreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-62131095218713280682014-04-04T16:46:53.199+01:002014-04-04T16:46:53.199+01:00@ILLY
The one thing that the BT/Unionist campaign...@ILLY<br /><br />The one thing that the BT/Unionist campaign fears most, is a gathering memento for the Yes campaign.<br /><br />This is why they would be happy to put out dodgy press releases suggesting Yes is languishing in the polls, because for all the possible down-sides to them from this strategy, it is more than made up, by stopping the gathering of a yes memento.Patrick Rodennoreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-29421764973287060302014-04-04T16:13:39.009+01:002014-04-04T16:13:39.009+01:00I think we're talking at cross-purposes. Obvi...I think we're talking at cross-purposes. Obviously higher levels of registration and turnout significantly reduces the importance of the correlation between past vote recall and referendum voting intention, but it doesn't mean that the correlation isn't there.<br /><br />RIP Margo MacDonald.James Kellyhttps://www.blogger.com/profile/01516007141763230886noreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-89449133704006126232014-04-04T15:46:17.822+01:002014-04-04T15:46:17.822+01:00James, My point is that with the referendum alread...James, My point is that with the referendum already delivering greatly increased electoral registration and promising the prospect of much higher turnout (about which I now agree with you) the voting pattern assumptions imported from past elections must be questionable (unless we assume that those who have self excluded from the electoral process did so confident that result determined by those who participated would faithfully reflect their wishes). To be clear, I am not saying that the assumptions are necessary wrong, merely that they hang upon the a rather shoogly peg.Alasdair Stirlinghttps://www.blogger.com/profile/01282612686734280239noreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-88364216516815094022014-04-04T13:48:51.630+01:002014-04-04T13:48:51.630+01:00Alasdair : To some extent I agree that we don'...Alasdair : To some extent I agree that we don't know, for the reasons set out in the previous post. But I'm struggling to understand how you can credibly argue that there may not be a strong correlation between voting history and referendum voting intention - the evidence that there is such a link is absolutely overwhelming. (Which is in no sense bad news - pro-independence parties outpolled anti-independence parties on the 2011 list vote, after all.)James Kellyhttps://www.blogger.com/profile/01516007141763230886noreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-88119060593660020742014-04-04T11:49:30.562+01:002014-04-04T11:49:30.562+01:00You expressed my thoughts better Alasdair.
I hope ...You expressed my thoughts better Alasdair.<br />I hope it is a massive Yes of course! :-)Jutemanhttps://www.blogger.com/profile/11061671774494923407noreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-32306004447526441922014-04-04T10:45:50.344+01:002014-04-04T10:45:50.344+01:00James,
Re: ‘Past voting history certainly correla...James,<br /><br />Re: ‘Past voting history certainly correlates strongly with referendum voting intention’<br /><br />We cannot rely on the datasets arising from referendum polling as evidence of a correlation between voting history and referendum voting intention. Remember that previous voting history is build into (most) referendum polling methodology (at the very least by means of political weighting, but arguably also by sampling by electoral constituency which bear no reference to the referendum). This dependence on historic voting patterns gives rise to a ‘fallacy of presumption’! That is: the proof of the conclusion (in this case that there is a correlation between voting history and referendum voting intention) rests upon the presumption of its truth (as evidenced by the pollsters including previous voting intention as sampling/weighting criterion).<br /><br />Even six months out from polling date, we have already a raft of evidence (voter registration, diverse campaign groups, social media activity and crowd funding initiatives, not to mention the amazing attendance at public meetings) that suggest that the electorate is taking this referendum into uncharted territory and we may fairly ask searching questions of any presumptions vis-à-vis past voting intention built into a polling sample.<br /><br />In short and to paraphrase Tony Blair: ‘the electorate is shaking the Kaleidoscope and the pieces are in flux’ and we may/must hold as highly suspect the reliability of any headline forecast of a Yes/No vote result reported by any poll relying on previous electoral patterns. That said, we do well to remember that this uncertainty cuts many ways. Perhaps the polls are correct and the No campaign is heading for a respectable/narrow victory. Perhaps the Ian Smart view/hope of the world is correct and cause of Independence heading for a massive hammering. Maybe the Yes campaign is heading for a massive victory on September 18th. The plain and scary truth is that we just don’t know!Alasdair Stirlinghttps://www.blogger.com/profile/01282612686734280239noreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-16772066457221423122014-04-04T09:17:56.781+01:002014-04-04T09:17:56.781+01:00I'm not trying to be picky, James,
I just feel...I'm not trying to be picky, James,<br />I just feel that something is wrong in the methods used by the polling companies for this referendum. I think the weighting might be at fault, as there is no past vote to weight. Asking folk what party they voted for last is only of limited use.<br />The actual result of this referendum may cause a few red faces amongst polling companies.Jutemanhttps://www.blogger.com/profile/11061671774494923407noreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-11994681696395911792014-04-04T07:53:00.818+01:002014-04-04T07:53:00.818+01:00A correlation between being over 65 and an intenti...A correlation between being over 65 and an intention to vote No, to be pedantic.James Kellyhttps://www.blogger.com/profile/01516007141763230886noreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-71143932942091398522014-04-04T07:32:00.600+01:002014-04-04T07:32:00.600+01:00"Not all over-65s will No, but there is a str..."Not all over-65s will No, but there is a strong correlation between being over 65 and voting No"<br /><br />How can that be? They have never voted 'No' in an Indy referendum before to base that assumption on.Jutemanhttps://www.blogger.com/profile/11061671774494923407noreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-17694399424990175392014-04-03T22:15:28.781+01:002014-04-03T22:15:28.781+01:00As another ex-Mori interviewer, I can say that my ...As another ex-Mori interviewer, I can say that my time working there was an eye-opening experience, and I will never trust a single result they put out again.<br /><br />If you have an understanding of demographics, you can see even down at the lowly level of phone-monkey that the quotas are often designed to steer the poll towards a certain outcome by focusing on age groups or socio-economic backgrounds that, statistically, will hold certain opinions. In one political poll, the preamble for the headline question was -five paragraphs long- and amounted to "X hates you, your family, your dog, and everything you like, love, and stand for; would you vote for X?".<br /><br />Now, Mori themselves aren't coming up with most of that kind of rubbish, the clients set the brief afterall, but the very fact they're willing to take on clients with such narrow, biased briefs clearly designed to generate specific results makes Mori as a company suspect in all regards, IMO.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-23862336886985937362014-04-03T20:31:11.021+01:002014-04-03T20:31:11.021+01:00Alasdair : Presumably you've seen the various ...Alasdair : Presumably you've seen the various datasets, so if you don't regard that as sufficient evidence of a strong correlation, I'm not quite sure what you'd be looking for?<br /><br />Juteman : Look at it this way. Not all over-65s will No, but there is a strong correlation between being over 65 and voting No. So if a poll doesn't have enough over-65s in its sample, ths Yes vote it reports is likely to be too high. By the same token, if you have too many Tory voters and too few SNP voters in the sample, the Yes vote is likely to be too low, regardless of the fact that not all SNP voters are planning to vote Yes and not all Tory voters are planning to vote No.James Kellyhttps://www.blogger.com/profile/01516007141763230886noreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-28507423272368202052014-04-03T20:19:14.858+01:002014-04-03T20:19:14.858+01:00Anon : The concern about public v private polling ...Anon : The concern about public v private polling is the question sequence. Clearly if Ipsos-Mori ask a leading question (indeed a downright inaccurate one) about Scotland being forced to leave the EU, and only afterwards ask the main referendum question, then they're not likely to produce an accurate result - the No vote will be too high and the Yes vote will be too low. That doesn't matter so much if it's a private poll - the client will be misled but that's the client's own fault. But if it's a public poll and Ipsos-Mori haven't disclosed what they've done, then that's a very serious matter, because it means we've all been led up the garden path. I find it hard to believe that they would do that, though, which is why I wondered if the description of the call sounded more like a private poll to you.<br /><br />Thankyou for explaining the whole process, though - it's really interesting to hear the details of how it works.James Kellyhttps://www.blogger.com/profile/01516007141763230886noreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-7081155816743027302014-04-03T19:28:19.595+01:002014-04-03T19:28:19.595+01:00Not particularly worried by the description, just ...Not particularly worried by the description, just a bit mystified that the survey didn't "screen out" right at the start. As to private poll versus one for publication, I see no reason why the methodology would change depending on the use that the results are to be put to - best practice is best practice - why change just because the client is going to keep the results to themselves.<br /><br />When I worked there these types of surveys were carried out using random digit dialing (RDD). A computer would randomly generate numbers, when it generated one with a suitable area code for the area being surveyed it would try to dial out, if it is an actual telephone number it will then cause someones phone to ring, when this is picked up it is automatically connected to a waiting interviewer who will introduce themselves and ask the responder if they are willing to take part in the interview.<br /><br />It is at this point, the start, that the demographic questions are asked - gender, age, ethnicity; why waste the interviewer's time (£££) conducting the interview only to find out the interviewee is in a demographic group whose quota is already full. Sometimes surveys did "screen out" half way through when a particular response was given; you would then have to apologise to a usually mystified person on the other end of the phone. I'm not discounting that they might be asking the age question at a different part of the interview - just that because it costs Mori to have interviewers sit asking people questions when they might not be able to complete the survey due to quotas being filled, you would expect that to be done right at the start.<br /><br />[Demographic questions asked to ensure a representative sample is included in the survey. Census data will provide the quotas for each demographic group.]<br /><br />I included the description of RDD as I think this excludes mobile phones from these types of surveys. I may be wrong - I was just a humble interviewer, and it has been a couple of years since I worked there. Hope that helps.<br /><br />ex Mori InterviewerAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-14887738489067044062014-04-03T19:18:56.309+01:002014-04-03T19:18:56.309+01:00Even some Tories are voting Yes, so I still can...Even some Tories are voting Yes, so I still can't understand how weighting works for the Indy vote?Jutemanhttps://www.blogger.com/profile/11061671774494923407noreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-83743020384797751452014-04-03T19:16:07.332+01:002014-04-03T19:16:07.332+01:00James, Do you have any evidence to support:
Past v...James, Do you have any evidence to support:<br /><i>Past voting history certainly correlates strongly with referendum voting intention</i>Alasdair Stirlinghttps://www.blogger.com/profile/01282612686734280239noreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-28078477416824933492014-04-03T18:28:45.328+01:002014-04-03T18:28:45.328+01:00Past voting history certainly correlates strongly ...Past voting history certainly correlates strongly with referendum voting intention, so that's the logic of weighting by it (although of course Ipsos-Mori are unusual in not doing that).James Kellyhttps://www.blogger.com/profile/01516007141763230886noreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-25408743485976340512014-04-03T18:25:06.421+01:002014-04-03T18:25:06.421+01:00I can't understand why/how weighting is applie...I can't understand why/how weighting is applied in a one off referendum?<br />Surely past voting history isn't so important?Jutemanhttps://www.blogger.com/profile/11061671774494923407noreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-42626064613474357032014-04-03T15:51:59.893+01:002014-04-03T15:51:59.893+01:00Just in case anyone misses it, I've now update...Just in case anyone misses it, I've now updated the post above with someone else's experience of being interviewed by Ipsos-Mori.James Kellyhttps://www.blogger.com/profile/01516007141763230886noreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-26191636790519826192014-04-03T14:44:20.585+01:002014-04-03T14:44:20.585+01:00Anon : Thanks for the clarification. So based on ...Anon : Thanks for the clarification. So based on your experience, should we be in any way worried by the description of the call given above, or does it sound to you more like a private poll?James Kellynoreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-1251165876145910182014-04-03T11:40:54.379+01:002014-04-03T11:40:54.379+01:00It is impossible for Ipsos Mori interviewers to as...It is impossible for Ipsos Mori interviewers to ask the questions in the wrong order. The order they appear will be determined by each particular survey; some rotate the questions to remove any sort of bias that the order of asking might introduce, otherwise they will follow a set order. Each questions appears on the screen individually in front of the interviewer and the next will not appear until the interviewer has typed in the response to the current question.<br /><br />ex Mori InterviewerAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-930120922627919768.post-91227315283305208192014-04-03T09:25:59.862+01:002014-04-03T09:25:59.862+01:00If the pollsters are manipulating/being manipulate...If the pollsters are manipulating/being manipulated, aren't they pushing it the wrong way?<br /><br />Wouldn't the better manipulation would be to have Yes be miles ahead, so that people think that they don't have to put in the campaigning effort, because it's already won?<br /><br />Rather than to have the polls show No farther ahead than it is, which will motivate the Yes campaign to campaign harder, and demotivate the No campaigners, because they think it's already won.<br /><br />I guess this is a carrot or stick thing, with the two options being "normailise but demotivate" or "try to convince them it's hopeless".Illynoreply@blogger.com