Apr 6, 2009

Using Email Domain Performance stats to spot ISP issues

Last week, I got a call from someone who wanted to switch from another ESP because he had a sneaky suspicion they were getting blocked by Yahoo too often (btw, DKIM can really help in this situation).

Blocks occasionally happen to everybody, so I told him we wouldn’t be immune. And believe it or not, ISPs have been known to have email problems themselves. What sucked about the guy’s situation was he couldn’t tell for sure if he was having a problem.

In MailChimp, if you want to know if your email campaign had troubles with any ISPs, you can always check your Email Domain Performance report:


In the stats above, 39% of all emails to comcast.net were bounced. It’s not a full 100% bounce rate, but it’s way above the average bounce rate for his industry. This particular user wrote me an email asking what they should do…

Since I know Comcast uses Cloudmark, I suggested they run an Inbox Inspection, because it includes actual scans using the Cloudmark spam filter. Then, they’ll know if it was a Comcast + Cloudmark issue.

If the email is extremely urgent, they could then send a followup email campaign (revised, based on their inbox inspection results) to their Comcast recipients by using MailChimp’s built-in segmentation.

The segmenting criteria would look like this:


since the user had MailChimp’s optional AIM reports installed, which tells you those who did NOT open or click (among other cool things), he could further segment his list like this:


You can see how adding the extra "those who didn’t open" criteria reduced the segment from 109 recipients to 75. Potentially, that’s about 34 fewer comcast subscribers who’d receive the followup email.

Two more tips for re-sending campaigns like this:

  1. Open tracking (and did-not-open-tracking) is not 100% reliable. Duh. So phrase your followup email in such a way that if people get dupes, it’s okay. Don’t just re-send the same email. They’ll get really annoyed, and report you for spamming. Change your design (good thing you already setup those 7 basic template types, right?) and write some copy like, "our last message seemed to have some issues with ___, so we’re re-sending it because it contains some urgent blah blah etc."
  2. If at all possible, wait a day or two before resending. If your campaign caused a problem with an ISP that got a lot of your emails blocked, the problem is not going to magically disappear in a matter of minutes. If you impatiently resend bad content, the only thing ISPs will see is, "Oh, big surprise. Bozo the clown over there, who got more than a 0.1% complaint rate for his last campaign, seems to be trying again. Time to reset the un-block-when-the-complaints-subside timer."
  3. If the email campaign is extremely urgent and you can’t wait, you should dramatically alter the content (including the design and coding) of the email to reduce the chances of any of that triggering spam filters again. Make it a plain-text-only email. Put the content on your website. In the email, point to that page on your website. I’ve even seen some users micro-segmenting their lists based on criteria like, "First name starts with a, b, c, etc." This will minimize your impact on ISPs, and help you spot further problems quickly.