Nov 21, 2008

Real stats: How sending to old lists will kill your deliverability

We have a customer with a relatively large list of about 311,000 opt-in subscribers. They’ve been collecting opt-ins from their site for years now.

About 240,000 of them are "old" (inactive) subscribers. About 70,000 are relatively "new" (active) subscribers.

They recently segmented their list and sent the same newsletter to each group (separately) over the same IP address, about 6 hrs apart from each other. Around 2pm, they sent the newsletter to the large, inactive list. Around 8pm, they sent the same newsletter to the active list.

The results are eye-opening…

Stats for the "Inactives" list (241,832 recipients):

Spam Complaints: 43
Open Rate: 6%
Click Rate: 2.4% (and 7,688 total clicks)
Unsubscribes: 264
Bounces: 6,878 (2.8%)

Stats for the "Actives" list (69,642 recipients):

Spam Complaints: 3
Open Rate: 36.3%
Click Rate: 7.4% (and 6,925 total clicks)
Unsubscribes: 96
Bounces: 128 (0.18%)

The number of bounces for the "old-members-list" is what I’d call very high. The number of abuse reports (43) is what I’d call disturbing. These are humans clicking the "this is spam" button for a run-of-the-mill company newsletter.

More interesting is what happened to their deliverability for the old list. We measured it using ReturnPath’s Mailbox Monitor service. Here’s a screenshot:

As you can see, their emails went 100% missing from AOL and Hotmail, 80% from Yahoo, and 70% for Comcast.

Also, it’s almost a day later, and 20% of the "sent-to-old-members" list still hasn’t been delivered to Yahoo, because they keep getting deferred.

My guess is the sheer number of spam reports from the old list is what got them blocked, and therefore their emails have gone missing (here’s some explanation from AOL on how they measure IP reputation).

If you’ve got a bad reputation, you’ll get blocked by AOL (and most of the major ISPs). Comcast’s Postmaster lists how emails get throttled based on your "Sender Score." The lower your score, the slower you’ll get delivered.

So how is your score determined?

AOL measures your IP reputation by:

  • Spam complaints (from humans clicking the "spam button")
  • Spam filter triggers
  • Bounces from undeliverable addresss (i.e. old lists)

I wish we had deliverability stats to show you how the "active-members-only" went, but we forgot to track the good one (d’oh). Maybe next time.

Even without a side-by-side deliverability comparison, the end result is clear: old lists suck.

They generate too many abuse complaints, which can get your entire campaign blocked by ISPs. Then, the active members who want your emails never get your message.

Related Stories:

Ever thought about segmenting your email campaigns so that they only go to the "active" members on your list? Here’s how to do that in MailChimp.