Sunday, September 14, 2014

Panelbase poll shows Yes at 50% among the whole sample

Many thanks once again to Ivor Knox for sending me the Panelbase datasets - and the big news is that the weighted results for the whole sample are even tighter than the published results (which are filtered by likelihood to vote).

Should Scotland be an independent country?  (Whole sample, Don't Knows excluded)

Yes 49.8% (+3.1)
No 50.2% (-3.1)

If those had been the headline numbers (and there are two pollsters that don't filter by likelihood to vote) they would have been rounded up to Yes 50%, No 50%.  In absolute numbers, there were 452 respondents after weighting who said they will vote Yes, and 455 respondents who said they will vote No.  Even on the raw unweighted data, which usually favours No more, the position is closer than I can ever remember seeing in a Panelbase poll - Yes 49.2%, No 50.8%.

Incidentally, on the published headline figures that take account of Don't Knows, the Yes vote stands at 46.1%, which is higher than in any previous Panelbase poll - and that includes the poll from last September which is normally disregarded because of an unusual question sequence.

As you'll probably remember, Panelbase made two methodological adjustments a couple of months ago, one of which had a Yes-friendly effect, while the other had a No-friendly effect.  The Yes-friendly one was the introduction of weighting by country of birth - the logic for which is inescapable, given that we know online polling panels have a disproportionately high number of English-born people on them.  YouGov have since followed suit.  But the No-friendly move towards weighting by recalled European Parliament vote is much more controversial, and isn't a procedure that is used by any other firm.  It looks absolutely certain that if the latter change hadn't been implemented, there would be a Yes lead in this poll, because the 268 people who recall voting SNP in May have been downweighted to count as just 220.

Turning to the ICM poll, the most frustrating thing about the datasets is that two different sets of fieldwork dates are given - we're first told that the poll was conducted only on Wednesday and Thursday, but then a period of Wednesday to Friday is given.  The latter version was the one reported by the Telegraph (which commissioned the poll), so hopefully that's correct - obviously the more up-to-date that a poll showing Yes ahead is, the better.

Of course when I first heard that a poll had shown the Yes campaign on 54%, the first thought that entered my head was that they had probably been flattered by the rounding.  But that isn't really the case - on the unrounded numbers, the position is Yes 53.8%, No 46.2%.  ICM weight their headline figures to take account of differential turnout, which in this case helps Yes slightly, but even without turnout weighting the numbers are healthy enough - Yes 53.3%, No 46.7%.

If the No campaign are scrabbling around in the datasets for some hope that the headline numbers may have been distorted, they might focus on the slightly implausible Yes lead in the South of Scotland sample, which has been upweighted almost two-fold.  But nothing else is leaping out at me - ICM presumably realised that they wouldn't get a satisfactory sample of 16-24 year olds, so in a repeat of what they did in a poll earlier this year, they've opted out of the problem altogether by weighting 16-34 year olds together.

Finally, there's big news about Opinium's methodology, which I'd suggest calls into question the credibility of the No lead they reported last night.  Unlike every other pollster, they've weighted their results to bring them into line with how people voted in the 2010 general election.  That's in spite of the very clear evidence that there is a particular problem with false recall in respect of 2010, with many voters wrongly thinking they voted SNP because of what they did a year later.  To be fair, Opinium have additionally weighted by 2011 recall, which dilutes the problem, but it certainly doesn't eliminate it - in this poll, 258 people who claimed they voted SNP in 2010 have been downweighted to count as just 179 people.  There's no real doubt that if Opinium had followed the same recalled vote weighting procedures that most other firms use, the Yes vote in the poll would be even higher than the 47.4% reported.

It looks like neither ICM nor Opinium are weighting by country of birth, which is a deficiency that in all probability is leading to the reported No vote being inflated.

84 comments:

  1. Good news as I think the polls are missing a large chunk of the Scottish demographic out, I don't believe they are reaching everybody and those they aren't reaching aren't pensioners with a golf club membership and a fondness for Last Night At The Proms, if you know what I mean. Apologies to Yes voting pensioners with golf club memberships of course :-)
    A poll is a poll, it's the voting that counts, roll on Thursday and let's hope we see Scotland rejoining the international community as an independent nation.

    ReplyDelete
  2. Even if the polls were telling the truth and the split is 47/53 or 50/50 differential turnout would still win it for either side.
    Hence the constant wetnat, concern troll, "don't bother voting, there's no point," bollocks.

    ReplyDelete
  3. massie on sunday day politics said the missing voters,newly registered would be voting NO


    any truth in that or a wild statement?

    thx

    ReplyDelete
  4. Tommy Sheridan says 60/40 yes.You know what I think he will be right.

    ReplyDelete
  5. His hubris reminds me of claims ahead of the 2012 Scottish local elections.
    He's still bitter that you ripped him a new arsehole over his claims that Labour won the 2012 elections.

    They’d swept to power at Holyrood with an overall majority the year before and Salmond and team repeatedly trumpeted that they were about to win full control of Glasgow. What happened? Labour returned to power with an overall majority and a night that should have been seen as an SNP success looked like failure.

    Mike Smithson (wank, wank, wank)

    ReplyDelete
  6. If you have not voted or are unregistered you are disenfranchised. Why would a disillusioned person refranchise themselves to vote no and maintain the thing they are disenfranchised from. Even a blind fool could answer that one. Its wishful thinking by Massie and Prof Curtice.

    ReplyDelete
  7. I dunno, I was talking to a guy yesterday - hadn't voted at all in 35 years but was registering to vote No. He didn't particularly care how the UK was run, just so long as there was a UK. utterly bizarre but there you go.

    ReplyDelete
  8. I suppose there is some twisted logic in that. However the unemployed and minimum wage people registering to vote are 80% yes according to most canvassers. If we go in neck and neck and take 200,000 of these people with us then its game over for no. No are clutching at straws as they know that they will lose if things go in as they are. Hence publishing an internal poll showing yes on 46% which we now know is bollocks. Then some no person tweeting they had a postal exit poll showing 60% no! Its not even remotely credible. They have nothing else left but self delusion.

    ReplyDelete
  9. Does anyone know when postal votes are openness

    ReplyDelete
  10. Varies depending on Council.

    My council opened on Mon 8th Sep.

    The council to the left of me opened weeks ago. 29th Aug rings a bell.

    But as far as I'm aware they don't get counted until Thurs 18th Sep with the normal ballot papers.

    Am willing to be corrected but that's my understanding

    ReplyDelete
  11. They have already been counted. They add them after the polls close at 10pm I believe.

    ReplyDelete
  12. I know a couple who have never voted before, claim voting achieves nothing and they are all useless.

    They are hardcore No!?!?

    ReplyDelete
  13. Checked on electoral website not entirely clear if postal votes are pre counted. Then put in the boxes sealed and sent for recounting after 10pm. I suspect they will have a double count in that case.

    ReplyDelete
  14. My aunty, never voted before is voting yes. As are many of the less well off. Ric had it at 60 40 for yes.

    ReplyDelete
  15. It was the Yes campaign and RIC who were most active in helping people who normally don't vote, to register to vote.

    They both said that some of the people indicated that they were registering to vote No, but that the overwhelming majority were voting YES.

    The postal votes should show a lead for No, as it would tend to be the elderly who require this method of voting the most.

    There will also be disabled who I would hope support Yes but may have been scared by the No scares, (if you haven't already please check wings for the story about the disabled woman who climbed up Edinburgh Castle and planted a huge Yes sign and left a message for Gordon Brown, She doesn't have long to live)

    We have got to win this folks.

    ReplyDelete
  16. Do they weight to the 2011 constituency or list vote?

    ReplyDelete
  17. James

    when are the next polls due out?

    thanks

    ReplyDelete
  18. I've no idea if newly registered voters would break for Yes or for No. But wouldn't they have been included in the polls anyway? Do pollsters only contact people on the electoral register?

    ReplyDelete
  19. Any disabled person voting 'No' is voting for at the very least a cut in their DLA when the new PIP system is implemented and the Tories are re-elected. Really they are being exceedingly stupid voting 'No'. I have no doubt my disabled wife will lose her car if Scotland stays with the UK.

    ReplyDelete
  20. Standard Q from me James...

    Scottish CoB % in the unweighted base?

    ReplyDelete
  21. SS : In the Panelbase poll, it's 77.7% (weighted up to 83.0%).

    ReplyDelete
  22. Another 2 undecideds for yes. That is 3 today.

    ReplyDelete
  23. Thanks James.

    Still struggling to get Scots to respond yet hitting parity.

    Nice.

    ReplyDelete
  24. Postal votes can only be verified before the count.

    Anybody reporting an actual number is either breaking the law or lying.

    ReplyDelete
  25. The more I see of your analyses, James, the less I need any part whatsoever of the MSM. They now have absolutely NO place in my life.

    ReplyDelete
  26. This has probably been answered already, so apologies in advance, but does anyone know what the timetable for polls coming out between now an Thursday is?

    ReplyDelete
  27. November 13th:

    There are two envelopes with a postal vote. The first envelope will have been opened.

    Inside is a second envelope which contains the ballot paper, it will remain sealed until the count.

    There is a voter signature on the outside of the inner envelope which will have been checked. If it doesn't match the signature that is on record for that voter then the ballot paper will be rejected.

    ReplyDelete
  28. Wee Jock Poo-Pong McPlopSeptember 14, 2014 at 7:43 PM

    Talking of polls (which we tend to do on this site, I realise...duh..) Are we to expect an Exit Poll on the stroke of Ten O'clock on Thursday Night? Usually happens with a General Election - and usually accurate too, IIRC.

    ReplyDelete
  29. It's going to be an exciting few days. Good progress for yes in the polls over the weekend, it's now hard to argue this is anything other than on a knife edge. Interestingly I think that no canvassers have not picked up the same swing on the doorstep; they seem pretty confident in their returns and the solidity of the no vote (in many cases having cautioned weeks ago that it was closer on the ground than some polls suggested).

    The big questions that remain unanswered are whether the polls have failed to reflect the impact of predominantly yes-supporting people registering to vote for the first time and whether differential turnout will favour no.

    And then there is the black swan question - does either side have a game changing trick up its sleeve. Hard to see that BT have anything of that nature in reserve, having thrown the kitchen sink at it over the last week. I doubt yes have anything up their sleeve either, but then I think the real focus for yes this week will be on creating a carnival atmosphere that encourages a huge turnout and persuades waverers they could be part of something special.

    Murdoch seems to have only recently realised Salmond is pro-EU. Very strange.

    ReplyDelete
    Replies
    1. I believe they Yes) have 2 very big cards to play....watch this space

      Delete
  30. Similarly to the points above I wondered on the day do we have any stats we should be keeping an eye on like for example if Edinburgh returns 51% No that would be a good result for Yes considering the expectation etc etc. Normally in elections there are some key constituencies whereby if you look at the results coming in it gives you an idea of how the night will go. Not sure if anyone has provides this kind of info anywhere yet.

    By the way as an Irishman I truly hope ye get the Independence you deserve and ignore the British establishment bullying. We did and although at times it was a struggle we wouldn't swap it for all the money in the world.

    ReplyDelete
  31. Flockers,

    Murdoch can, quite frankly, get to f*ck.

    He's playing the game, unelected power, "look how much people hang on my word still".

    And the Establishment media all play along.

    Corrupt and vile Australian American trying to tell Scots what to do. Ignore.

    Hugh.

    ReplyDelete
  32. Murdoch and Henry McLeish can both gtf. Both attention seeking and teasing.

    We do not need them.

    ReplyDelete
  33. Latest weird tweet from Murdoch

    After stating Scotland is ready for independence emotionally and politically he adds this:

    MURDOCH QUOTE "SNP not talking about independence, but more welfarism, expensive greenery, etc and passing sovereignty to Brussels."

    You do wonder how he could not have known this. Have there been further discussions? But hard now to see now he could put Sun on board though you can never quite tell, a guarded endorsement along those lines just possible I suppose.

    BTW anyone here who's champing to be rude to him back, I can tell you the replies to his tweets hold nothing back and are often unprintable. Many tell him to pass on in every imaginable sense.

    ReplyDelete
  34. Maybe Murdoch is wanting to declare for no but is feart that he'll lose half his readers?

    ReplyDelete
  35. Expat

    If Murdoch "endorses" something, whether it's No or Yes, the only rational reaction is to do the opposite.

    What's in Murdoch's interests is, without fail, against the interests of ordinary people and democracy.

    Hugh.

    ReplyDelete
  36. Anyone fancy shedding light on the betting odds?

    Polls are tight (apparently...) but the bookies would have you believe that No was about to win by a landslide.

    Not a gambler; any opinions would be helpful. It wrecks my positive mindset when I look at Oddschecker.

    ReplyDelete
  37. I think like better together and london the bookies are expecting people to bottle it like Quebec.

    Scotland is no quebec.

    ReplyDelete
  38. Mark,

    The bulk of money from bets on politics come from trader type City Tory cocks in London with more money than brains, always have.

    They're bound to be on No, as Scots are too wee, too poor and too stupid, surely?

    Just take a look at political betting.

    (incidentally maybe I'm just "not getting it", or "don't have a sense of humour", but the latest thread over there looks like a racist bag of filth to me)

    The value bet is very clearly on Yes.

    Hugh

    ReplyDelete
    Replies
    1. Thanks Hugh.

      I'd heard about some Londoner slapping £800k on a No result. I think it said he'd make less than £200k profit, but I don't doubt it's all loose change anyway.

      I'll have a look at Political Betting - thanks.

      Have you seen anything about data supposedly published by William Hill that was able to break down Scottish towns and cities and their percentage yes and no bets? Dundee and Motherwell sat at 100% yes while Glasgow was c80% yes and Edinburgh 50% yes.

      Surely not every bet in Dundee and Motherwell back Indy? I know Dundee is yes daft, but 100%...?

      Delete
  39. Mark
    This is something I posted on another site. Don't know if it helps (in response to someone who was asking why such goods odds for Yes were available)
    :

    Political betting is a strange beast. In any other betting field punters bet on their best guess at an outcome with a view to making money. In this market however people, generally, vote for what they want to happen rather than what they think might happen (although the two may coincide).

    But this time, although many more individuals have bet for a Yes, a few individuals with a lot of money have voted No, effectively distorting the market. The bookies are not concerned who wins, only in balancing their books to try to come out on top whoever does (their margin). Usually, they win more when the outsider wins because of an imbalanced book with too much money on the favourite.


    In short, the odds don't reflect the reality. People look at the odds though and think the outsider can't win because the bookies price says so. That's why current odds are so attractive.

    ReplyDelete
  40. Mark - I agree with the previous comment on betting and odds. I'm no expert but I understand that Bookies aim to make money either way - ironically they are not gamblers !. The odds dont reflect necessarily their assesment of the probablities. As we know a few punters have put huge stakes on NO which probably skews the odds as the bookies are trying to encourage money onto the YES bet. In effect the NO bet is overpriced and the YES is discounted thats why it's the value bet. More interesting is the number of bets going on each side which significantly in many Scottish cities a lot more bets are going on YES (albeit much lower stake per bet)

    ReplyDelete
  41. Wee Jock Poo-Pong McPlopSeptember 14, 2014 at 9:15 PM

    Thanks, Ian Baird, for that explanation. I'm not a betting man either so was completely foxed by all this. Intriguingly, I suddenly cannot get anyone at my place of work (in England, sadly, so my support for Yes is ineffectual)to take any sort of bet on the result. They seem to be waking up o the idea that Scotland might no longer be "theirs" next week!

    ReplyDelete
  42. ODDSCHECKER

    Best on yes 7/2 but most 3/1 only one 11/4

    Best on a no vote is 1/4 but most either 2/9 or 1/5

    ReplyDelete
  43. Thanks for all the replies RE betting. Much appreciated and has gone some way to restore my faith in Thursday night.

    Extra wee bonus for a yes vote will be the upset Tory boy in London who's £800k lighter. :-)

    ReplyDelete
  44. Some rumours of a "big name" coming out for No.

    Any guesses? Princess Anne?

    ReplyDelete
  45. James Kelly,

    Doesn't seem you're banned from PB any more. A poster who (I think) is a serious Mod just suggested so.

    This of all weeks, (if you're not too busy like!) the site may value your input!

    If you want.

    Hugh

    ReplyDelete
  46. Do we know when the next poll will be released?

    ReplyDelete
  47. Wee Jock Poo-Pong McPlopSeptember 14, 2014 at 9:54 PM

    Will be listening to "The Westminster Hour" on BBC R4 in a few minutes. With any luck, some panicky Establishment types bleating...!

    ReplyDelete
  48. Stickers
    Princess Anne may come to Scotland to watch the rugby team but she's part of the establishment so it would hardly be a surprise.
    But Prince Harry declaring for No might increase the Yes vote.
    Can't think of anyone that would sway voters to No at this stage .

    ReplyDelete
    Replies
    1. Cant be Jesus. He is a proper socialist and a Yes voter.

      Delete
  49. "Can't think of anyone that would sway voters to No at this stage . "

    Dig up Thatcher and wheel her around.

    www.independent.co.uk/voices/comment/sending-the-least-popular-englishmen-to-plead-with-the-scots-that-was-a-good-move-9727290

    Hugh

    ReplyDelete
  50. A big name

    Jan Venegoor of Hesselink

    Its probably Gordon Ramsay.

    ReplyDelete
  51. Wee Jock Poo-Pong McPlopSeptember 14, 2014 at 10:04 PM

    Oh yeah..H.M. the Q. is the "Big Name".

    Personally I prefer my royals to be a bit less German...but seriously, will this help the other side?

    ReplyDelete
  52. Do you mean "Yes" have cards to play?? Your message is a little ambiguous.

    ReplyDelete
  53. Is this rupert entering the fray for No?

    The only other big name would be andy murray

    ReplyDelete
  54. According to the Times it's David Beckham. Cannot imagine that having much impact.

    Andy Murray has already pretty much said he is no (although can't admit it publicly for fear of backlash).

    ReplyDelete
  55. That might go against his mums wishrs....


    Anyway, survation released their data tables for ruk mail on sunday poll....it does not bode well for us if its a no vote. We will seriously be fucked over, not feeling the love LOL

    ReplyDelete
  56. Any clue about next polls?

    Best,

    Xabi

    ReplyDelete
  57. I had not realised at all that the ICM poll with the 8% lead for YES was called an outlier not just as a possibility by James but by the boss of ICM Martin Boon.

    ICM QUOTE

    Martin Boon, the head of ICM Research, said his firm's poll should be seen in the context of a volatile campaign in which results have shifted dramatically.

    “The proper way to analyse this poll is simply to take it as an ‘outlier’ which fits into the overall impression created by all Scottish polls right now: it seems too close to call, and could go either way.”

    Is this caution, bias, reality or what?

    ReplyDelete
  58. It's interesting that Boon has spoken out against his own poll. However, I think he's likely being very cautious at this point, and I don't doubt the figures the ICM poll released.

    ReplyDelete
  59. Anyone know when the next polls get released?

    Good turn out today in Aberdeen rally. Not feeling so good about the overall vote up here though. Anyone know how Aberdeen is doing in the polls?

    ReplyDelete
  60. big name supporting NO is David Beckham

    walofs

    ReplyDelete
  61. That might go against his mums wishrs....


    Anyway, survation released their data tables for ruk mail on sunday poll....it does not bode well for us if its a no vote. We will seriously be fucked over, not feeling the love LOL

    ReplyDelete
  62. That might go against his mums wishrs....


    Anyway, survation released their data tables for ruk mail on sunday poll....it does not bode well for us if its a no vote. We will seriously be fucked over, not feeling the love LOL

    ReplyDelete
  63. Wheres James? It's like waiting for Batman to appear.

    ReplyDelete
  64. Andy Murray is not a No when he was asked who he would play for if Scotland was independent he said 'Scotland' he then said that it didn't look like it would be a Yes (this was when the polls had No well ahead)

    ReplyDelete
  65. When Andy Murray first came on the scene he was was wearing sweatbands with the Scots Flag on it, also I saw him in a press conference questioning why Scotland and England play football, rugby, golf, cricket, snooker, darts, etc etc, as separate countries but at tennis we play as Britain ???

    Plus he sarcastically took the mickey out of the English Football team about 8 years ago '' I'll be supporting anyone but England,'' This was reported by the London press and he never fully recovered from the backlash, public relations wise, they've painted him as a '' dour boring Scot.'' That's probably why in interviews now he just gives bland statements because he doesn't want the media jumping down his throat for any slip ups he might make.

    Of course Andy Murray has always been popular in Scotland with a dry sarcastic sense of humour, which the English don't seem to get.

    I suspect he's an Independence supporter but he can't come out for fear of the London media industrial complex crucifying him again.

    ReplyDelete
  66. Apart from David Bowie, Andy Murray must have the lowest ratio of words spoken about the referendum to analysis in the media of those words.

    ReplyDelete
  67. Isn't there a problem with purely online polling in that you have to be both online and signed up to a polling agency? Does this not automatically exclude older, small c conservative types who might not be particularly taken with the internet or computers? I realise the under represented demographics are 'upweighted' - but what if internet use represents a demographic in itself?

    ReplyDelete
  68. No matter how many times Englands National team has failed in major competitions, I could always get very generous odds on the other team winning, just because few punters would vote against their own side.

    I would think that this bodes well for Yes, if people are backing their own side in the bookies.

    Dundee did indeed have 100% for Yes, but I think this was only with William Hill, who released their betting data. I don't think any other bookies have released their data, but not sure?

    We only have a couple of days until the polling stations are open!

    I'm filled with every emotion I have ever felt, and some I didn't know I had, lol!

    ReplyDelete
  69. Should have added that I got generous odds for teams playing England, because I was living in England at the time.

    ReplyDelete
  70. Given that the polls are tight (which I do not really believe) then we get into the realms of other factors
    Now I seem to remember that there was two factors which helped yes
    The Yes voter is more likely to turn out ,and the bigger the turnout the better for yes
    IIRC curtiss said this could be worth 3 or 4 points to yes ...comments ?

    ReplyDelete
  71. Hey, James on BBC breakfast tv, good stuff! Appearing with Duncan Hotherstall over and above the call of duty.

    ReplyDelete
  72. David Beckham? FFS! Who thinks up these anti-No gestures?
    I look forward to hearing his speech, how concerned he is for the unemployed and the users of food banks, and people having their benefits cut..
    But we are Better Together David?
    PS Loved that penalty you missed when you went on your erky...

    ReplyDelete
  73. In case it is of your interest, Spanish newspaper El PaĆ­s has today an editorial on the Scottish referendum titled "Scotland is unique".

    http://elpais.com/elpais/2014/09/14/opinion/1410719435_239400.html

    The article endorses no one, but generally commends the pro-independence side for its civility, for its respect for constitutional legality, and for posing reasonable, prospective arguments, instead of purely emotional and retrospective ones.

    Xabi

    ReplyDelete
  74. Historically, Yes support shows a very good correlation by region against SNP 2011 regional share. As you might expect.

    Since 2007 that pattern broke down at times, with odd results in that e.g. nationalist heartlands suddenly were bastions of the union.

    The pattern returned again in 2009 when it was clear there'd be no referendum and the heat came off Scotland. It broke down again 2010-early 11 when the heat was back on, then reappeared in the glory of 2011.

    It broke down again 2012-13 and into 14.

    It's back in the ICM which has Yes ahead. Which is why Yes went ahead.

    ReplyDelete
  75. As you have mentioned Skier...'national identity' coming to the fore....I hope the Yes message this week is one of history making and for Scotland.

    Appeal to the heart and people will find it very difficult in the polling booth to vote No for Britain.

    ReplyDelete
  76. Xabi, there's plenty on here who will argue London-based papers push London-based agendas. Given that, it is not more than likely a Madrid-based paper is pushing a Madrid-based agenda i.e. try and stop the catalonians having their own referendum - "its respect for constitutional legality"?

    ReplyDelete
  77. As you have mentioned Skier...'national identity' coming to the fore.

    Playing a role definitely.

    Large group of Scots (born) still refusing to be polled, but this seems to have been reducing and related to sudden poll movements. Yes amongst Scots born shooting up.

    Lots of anecdotal reports of Nos moving to Yes due to ramp up of scare stories, BBC stuff etc.

    ReplyDelete
  78. ylee coyote said...

    Actualy, it's not quite true that a high turnout faviours Yes. Well, a high turnout favours Yes but a very high turnout favours No. As you say Yes voters are more committed to vote. Let's say there is a 75% turnoutout. That could mean that some No voters have stayed at home BUT if the turnout is 85% it means that both the Yes and No camps have got their vote out. The higher it is is the more No votes have been given lifts to the polling stations and this will work against Yes.

    ReplyDelete
  79. Skier, I've seen it with my own eyes, die-hard unionists, have turned to Yes due to the media barrage.

    It's unbelievable.

    ReplyDelete
  80. Have you been infected by the infamous liar and PB moderator TSE? Putting Carly Simon lyrics in the thread headers I see.

    ReplyDelete