Apr 11, 2008

Hard Bounces Reveal Signs of Trouble

I just sent a campaign for MailChimp yesterday, and checked my stats. Here’s my personal stat-checking routine:

  1. Check for abuse complaints. My list is double opt-in, and I never import anybody, so complaints should be virtually zero. I got no complaints. Hooray!
  2. Check the feel-good stuff, like open rates and click rates. I rarely send emails to this list (don’t want to bug them) so I typically get between a 40%-50% open rate. Yep, 41% so far.
  3. Look at what URLs people clicked on. I like to see what my audience is most interested in. It’s usually techie/how-to stuff.
  4. Check bounces.

Bounces are where a lot of people get confused, because there are different types of bounces, and different ways you should treat them. But they also reveal a lot about your list, and can be like that canary in the coal mine, telling you "something’s wrong and you better act soon."

Soft bounces are when an email address is temporarily unavailable. Maybe the recipient’s server was down, or just too busy. You generally don’t have to do anything—MailChimp retries delivering a reasonable, non-server-annoying amount of times, and if it still gets rejected, we keep that email address on your list for the next campaign. It’s only if an email address soft bounces 5 campaigns in a row that we clean it off your list for good.

Hard bounces mean an email address was non-deliverable. It’s gone. Deleted. Doesn’t exist anymore, never existed in the first place, whatever. Lost cause. These get removed from your list by MailChimp immediately. That’s because if you try resending to these emails, you could get blocked by that server (they track who keeps sending emails to bad addresses).

In general, hard bounces are more important to look at than soft.

So when I check my stats, I go to the hard bounces and I look for trends:

  1. Did I get an abnormally high number of hard bounces? Half of one percent of my list hard bounced, which isn’t too shabby (here are average email bounce rates by industry).
  2. Visually scan the domains in my hard bounces. Are there lots of hard bounces from the same domain? This tells me if any particular ISPs or company email servers are blocking me (which in turn would lead me to contact their postmaster to politely request unblocking).

For example, my hard bounce list looks extremely random. Every email that bounced was from a different domain, and none of those domains were from a major ISP (like hotmail, aol, yahoo, gmail, etc).

On the other hand, if you see a block of hard bounces from the very same domain, this would indicate there’s a problem to look into. Let’s say you got 10 bounces, all from comcast.com addresses (this happened to lots of people back in December 2007). First, we recommend you take a look at your content, to determine whether or not it had things that trigger spam filters. Don’t just arrogantly assume, "I’m not a spammer, and this isn’t spam, so it can’t be my problem." Most of the time, it’s your content, not your email delivery provider. Or, it’s your list (sending to a really old list can result in blocking, especially if you’ve got a spam trap on the list).

But if you see that your content is just fine, or if it’s a really overwhelming number of hard bounces from one domain (and no other emails to that domain got through) then we definitely have a problem, and it’s time to call your email delivery service.

Don’t just focus on your open rates. Hard bounces are your early indicator of deliverability problems.