Hi Nigel, On Tue, Jul 30, 2013 at 05:39:40PM +0100, Nigel Rantor wrote: Surely Hotmail and Gmail would be best placed to describe their own policies on email filtering? Yeah, I know: they probably just ignored you or pointed you to a knowledge base document that did not answer your question. That's probably the best you're going to get, I'm afraid. Absent an actual blacklist report anything else is just technofolklore - when we pray to the email God and sacrifice the black goat, email is deliver good. This thread was on a similar topic and had a similar non-outcome: http://lists.bitfolk.com/lurker/message/20130626.122040.2e8cbe5c.en.html Cheers, Andy -- http://bitfolk.com/ -- No-nonsense VPS hosting

I've been dealing with email deliverability recently (as Andy linked in his response). Rather than being explicitly listed on a blacklist, it's more likely that your B domain hasn't yet built up enough of a reputation (but domain A has a reputation, hence why those emails are delivered). They take all sorts of stuff into account when determining what to junk including domain age, email volume and history, IP reputation, etc. Just ensure you do everything you can on your end including SPF, DKIM/Domainkeys, appropriate HELO, reverse DNS, sign up to feedback loops, relaying mail via SES or Mandrill helps if you only send low volume, take care with how emails are worded, it can also just take a bit of patience. Regards, Luke On 30/07/2013 17:39, Nigel Rantor wrote:

On 30/07/13 17:46, Andy Smith wrote: Well, I already started looking at this on gmail a few weeks ago, and yes, I couldn't find anyone, just vague support articles. Looking again now though I see a new thing in their support sections, DMARC, which I had not heard of before. Thanks, I will be rooting around and trying any/every thing I can find from the list and other sources. As a bit of an update it *might* be the case for gmail that after having 4-6 friends unmark messages from the domain as spam other people who have never seen the domain in the inbox are getting delivered directly now...which is nice...I will keep the list up to date if I find anything interesting. Regards, Nigel

The really interesting thing about that tool is that it says there is not SPF record for alertferret.com...which is complete lies because there is; i.e http://mxtoolbox.com/SuperTool.aspx?action=spf%3aalertferret.com&run=to… May change it to the suggestion and see what microsoft's page thinks afterwards. Thanks, Nigel On 30/07/13 17:51, Murray Crane wrote:

Further to on-list discussions another sysadmin friend of mine suggested that the fact that the A record for my MX doesn't match the PTR record might be an issue but I feel it is a red herring. I'm pretty sure multiple PTR records is not allowed (or at least discouraged) anyway, which means I would need to start using the real name of my machine for the MX record instead of the one I am using. This isn't that much of a problem really, just means I don't have a nice 'mail.*' MX entry. Thoughts? n

I may be on to something...examining the raw message in gmail shows the following; Received-SPF: fail (google.com: domain of wiggly(a)wiggly.org does not designate 2001:ba8:1f1:f03f:216:4eff:fe05:ae0f as permitted sender) client-ip=2001:ba8:1f1:f03f:216:4eff:fe05:ae0f; Authentication-Results: mx.google.com; spf=hardfail (google.com: domain of wiggly(a)wiggly.org does not designate 2001:ba8:1f1:f03f:216:4eff:fe05:ae0f as permitted sender) smtp.mail=wiggly(a)wiggly.org I have ignored the whole IPv6 issue until now and I'm not sure what my options are. I'm about to go do some research but I guess it is one of; a) stop responding to IPv6 b) fix DNS and PTR records to have correct IPv6 address *wanders off to read gandi DNS docs* n On 30/07/13 21:13, Robert Gauld wrote:

On Tue, Jul 30, 2013 at 11:37 PM, Nigel Rantor wrote: This explains. The -all in your SPF record says that you will only send email from the domains (in your case the A and MX records) that are listed in your SPF record and you give recipients implicit permission to discard emails pretending to come from you that fail SPF. Not many recipients do that, because many domains use -all incorrectly, but it means your emails will get a lot of spam-points added and they're going to be blocked unless there is a good reason for not blocking them. A 19-year old domain with a good reputation could be such a reason and that's likely what made the difference here. The simplest solution would be to add your IPv6 address (or, perhaps, your /64) to your SPF record as ip6:2001:ba8:1f1:f03f:216: 4eff:fe05:ae0f. Or stop sending email over IPv6 of course. Martijn.

On 30/07/13 23:50, Martijn Grooten wrote: Yeah, the easiest for me was to tell exim not to use ipv6 for now. First email I tried via mail client got through okay. First email I tried through the automated system that runs behind alertferret.com failed but the message displayed in gmail as to why this message was marked as spam was different, it now says it is spam because it is like other messages that have been marked as spam. So, I deleted all the spam messages and sent another...still being classed as spam. Bugger. I am going to delete all email from the gmail test account I am using and try again tomorrow maybe, just in case...I am worried though that it will now have 'learned' that the domain is poisoned... n

On 31/07/13 00:43, Nigel Rantor wrote: [snip] I recently had much the same problem and for the same reason. Gmail - having been very good about not mis-classifying e-mail as spam - suddenly started filing all stuff from my domain as spam. The reason as it turned out was because the e-mail was indeed going via IPv6 and my SPF record didn't list my IPv6 address. In the course of investigating I found a very useful site. Send an e-mail to check-auth(a)verifier.port25.com and you get back an automated acknowledgement telling you how well it did in security checks. It covers SPF, DKIM and other bits. I both fixed my SPF (by adding an IPv6 record to my host name) and added DKIM signing to my outgoing e-mails. Google is now a happy bunny again. $ host -t txt sinodun.org.uk sinodun.org.uk descriptive text "v=spf1 a:ntp.sinodun.org.uk a:nimbus.sinodun.org.uk ~all" $ host nimbus.sinodun.org.uk nimbus.sinodun.org.uk has address 85.119.83.78 nimbus.sinodun.org.uk has IPv6 address 2001:ba8:1f1:f187::2 nimbus.sinodun.org.uk mail is handled by 10 nimbus.sinodun.org.uk. $ HTH John

On 30/07/2013 18:21, Jan Henkins wrote: If you think that's bad, then consider that there are a significantly non-zero number of domains that actually explicitly use +all, which as far as I'm concerned is tantamount to declaring 'all spammers in the world may use my domain freely'. http://spf-all.com/stats.html In my mail setup I normally allow domains that pass SPF to skip greylisting, but I have an explicit check for +all which treats it as neutral and falls through to greylisting. Interestingly, I'm starting to see greylisting become less effective as an anti-spam technique. Traditionally greylisting works by temporarily rejecting mail on the first delivery attempt on the assumption that genuine MTAs conform to the specs and will attempt to resend a short while later, while spambots will simply give up and move on to the next target. A few years ago nearly all of the small amount of spam I got was from hosts which passed SPF (presumably either through compromised systems or through loose SPF settings). Nowadays most of my spam is SPF neutral, which in my mail setup gets greylisted. This means that these spams are getting through greylisting, so there are now some spam networks out there which have clued on to this. -- Phil

Phil Stewart said: That's interesting. I'm not noticing any increase in spam getting through, and the logs still show an awful lot not getting past greylisting. There are some 'genuine' senders who don't either because their mail servers don't resend the mail, but these tend to be the ones who don't accept postmaster@ email either, sigh. Ian

Hi Nigel, On Thu, August 1, 2013 11:30 am, Nigel Rantor wrote: For what it's worth, a *new* record should effectively be propogated straight away assuming your DNS is configured correctly (i.e. slaves set to update immediately following changes to the master). Mathew (Keen to see IPv6 usage increase, not decrease!)

Thought I would post a final update regarding all of this and thank everyone for their thoughts, help, pointers, resources and time. In the end there were two problems that appear* to have been causing mail to be flagged as spam. 1) SPF / IPv6 I set up SPF records a while ago, maybe a few months, when I got the alertferret domain and rejigged my exim config to allow me to add email domains to my server by just having config files** added by puppet. I did not realise at the time that my server would attempt to contact other people's servers using IPv6, I thought that IPv6 was only in use within bitfolk's borders. I have not been keeping up with IPv6 rollout/adoption so this is my bad. Since there are no AAAA entries for my machines nor PTR records when my exim server spoke to google's mail servers it failed the SPF test and got canned as spam not ham. Fair enough. Current fix is to stop exim using IPv6, I now need to go and look at my machines IPv6 setup at some point. This would not, I assume, have been a problem if I were using a 3rd party email provider like sendgrid/mandrill. 2) Content The alertferret domain is used by a little service I wrote which sends messages on behalf of the user. The messages were taking advantage of "Reply-To" to ensure that if the recipient just hit "Reply" on their MUA the message would be sent to the user who created the message content, not the "noreply(a)alertferret.com" address used as "From" address on the emails. Removing this stops the google/hotmail systems from canning the messages as spam. This would not have been fixed by using a 3rd party email provider like sendgrid/mandrill. Conclusions I would like to be able to use Reply-To as it was intended, and I think my use is perfectly legitimate, the messages themselves make it clear that the message has been sent on behalf of someone else, and clearly states the person's email address. Unfortunately spammers have messed it up for everyone by making the "Reply-To" header a signal for spam. I may have to add an auto-responder to the "noreply(a)alertferret.com" address to inform the person that they should maybe reply to the email in the message rather than the "From" address. Also, having a 19 year old domain that has always been well-behaved w.r.t email and is whitelisted by a few resources is great but I think I need to get a new throwaway domain to test email things in the future. Thanks, Nigel * since this is still all a bit voodoo and unknowable because I cannot ask G/MS which items actually were responsible for getting flagged as spam ** user and alias lists

On 01/08/13 13:56, Andy Bennett wrote: Ah, I omitted this on my resolved update. I had A otter.wiggly.org -> IP A mail.wiggly.org -> IP MX mail.wiggly.org IP PTR otter.wiggly.org So when doing a full circle you found that the MX name didn't match the PTR record for the IP it resolves to. I fixed this before finding the problem with the SPF / IPv6 issue so I don't know if it was another problem that required fixing, it is better to have these the same though so I am leaving it that way and will have to live with my MX being called "otter" and not "mail". Regards, n

On Fri, August 2, 2013 12:11 pm, Martijn Grooten wrote: I agree. Otherwise, you'd run into all sorts of difficulties where a mail server is responsible for a number of domains and you desire it to be referenced (through MX records) via a name unique to each domain. The PTR record can only point to one of them and the others shouldn't be expected to suffer as a result. Mathew

On 02/08/13 11:15, Nigel Rantor wrote: AIUI, This arrangement is fine provided otter.wiggly.org appears in the HELO repsonse at the SMTP level. It's must be: think about how it might work if you had more than one relay. Also, look at the DNS setup for cam.ac.uk: that's a relay that's known to be well run. Regards, @ndy -- andyjpb(a)ashurst.eu.org http://www.ashurst.eu.org/ 0x7EBA75FF

Months ago Simon said: Thanks for that - they don't exactly make it easy, do they? Earlier this week, moved email servers for the family email and I've just got the bounce from £$%^ing Yahoo... for a sky.com email address. The IP isn't in any public blacklists, it's one of about four emails the new server has ever sent to a Yahoo account, the SPF for the domains are set up and it's still going "All messages from xx.xx.xx.xx will be permanently deferred". It has got me to set up DKIM at last, and verifier.port25.com thinks it's ok. And despite this sort of crap, hacked Yahoo accounts successfully send me more spam than anything else. Motes and beams, motes and beams. Ian

John Winters said: Quite. To add to the annoyance, they moan about people resending mail they're not going to deliver, but they reject it with a 421 (Service not available) temporary error rather than a 5xx permanent one. Even their own 'help (sic)' pages say a 421 error means.. "Resources temporarily unavailable. Please try again later: This means that the Yahoo Mail servers are temporarily too busy. Your server should try to resend the message after a few minutes" Ian

Jan Henkins said: I greylist too (see past posts here) and I find it to be more effective than that. But this is Yahoo using a 421 error to refuse to accept mail permanently - see the error message they actually send: "421 4.7.1 [TS03] All messages from x.x.x.x permanently deferred. Retrying will NOT succeed. See http://postmaster.yahoo.com/421-ts03.html" .. and they mean it. (Even if they get the URL wrong.) No matter how many times it is resent, or how long is between resends, it is not accepted every time. This isn't greylisting, this is blocking replies to their own users' emails. I can't even reply to emails I send from my own Yahoo account. Oh, if I try sending an email to mail-tester.com, I get a 9.8/10 score - the 0.2 deduction being because the text of the email ('test') doesn't look like a normal email. SPF pass, DKIM pass, reverse IP lookup pass etc. Ian