Even though these reports center around personal webmail services and personal social accounts, ESP customers should look into beefing up their security as well.

Over the last couple years, attacks on ESPs have been on the rise. Sometimes, the ESP has been breached, but more commonly it’s an end user (or the end-user’s marketing agency) that unwittingly gives their account credentials to spammers (usually via malware on their computers).

So here’s a not-so-gentle reminder to MailChimp users, and especially MailChimp users who manage multiple accounts: GO ACTIVATE OUR SECURITY FEATURES.

In case you missed the recent announcements:

• Alter Ego 2-factor Authentication
• Security TXT Alerts
• Location-change security question

New Feature: Generate Strong Passwords

The idea is simple.  You create a unique disposable email address for each newsletter or web service you use.  If that address starts receiving email from a disreputable marketing service, you’ll know your information has been shared and who shared it.  Should that happen, you can either delete the alias address or set a filter on the incoming email.

It’s the kind of oversight that paranoid and vindictive people like me just love, but it’s also an additional layer of protection between you and spam no matter how curmudgeonly you are.  Let’s go over the details on MailChimp’s top three ISPs.

Gmail

This may be the easiest disposable email service of the three, but in my opinion it’s also the most useless.  You can take any gmail address and add a plus sign and a keyword to create as many disposable email addresses as you want.  You don’t have to register them with Gmail first, and they’ll deliver to your regular address automatically.

For example, let’s say I own example@gmail.com.  I could create the following disposable email addresses on the fly at any sign-up form:

• example+newsforum@gmail.com
• example+bookclub@gmail.com
• example+nospam@gmail.com

The keywords after the + sign let me keep track of where I used each disposable address.  If I start getting non-bookclub emails at my bookclub address, there will be hell to pay.  The problem is that my actual gmail address is still highly visible.  Spammers know how to circumvent this plus sign stuff, so it doesn’t offer a lot of protection.

Windows Live Hotmail

This takes a little more effort to set up, but you’ll be able to hide your primary email address.  That’s because Windows Live lets you choose an entirely new ID.  It’s called an email alias.

Finding this feature in your options can be a little challenging.  To be honest, I never found it, but I did stumble across this helpful link which saved both my head and the wall from needing extensive repairs.  Just a warning, if you want to look through the Windows Live Hotmail Help Center, make sure you have ice and touch up paint on hand.  Okay, moving on.

All you need to do is think up a unique alias name, and register it through the form linked above.  You can allow this address to come to your inbox, but to Hotmail’s credit, they make it really easy to redirect this mail to a folder.

You can create 5 email aliases a year, but they cap you at 15 total.  This is really the biggest downside.  As you sign up for more and more newsletters, blogs, forums, and retail sites, you may not be able to give each one a new alias.  Should this happen, remember that Hotmail also accepts the on the fly “+” nomenclature.

Yahoo!

Yahoo also offers email aliasing, and I really like their service.  You can have up to 500 aliases, and they force you to use a new basename and a keyword that aren’t your primary email address.  The only drawback is that you have to be a Yahoo! Mail Plus user, and that is not a free service.

For those who are Mail Plus users, just go to Option and then More Options.  You should see a menu item for disposable email addresses.  This form makes it easy to add, edit, and remove any disposable email addresses you create.

As I mentioned before, they have separate fields for the basename and they keyword.  All Yahoo email aliases will be in the following format:

• basename-keyword@yahoo.com

The basename has to be a unique Yahoo ID, but the keyword is there to help you remember where and how you are using this alias.  Name it wisely, by which I mean, be obvious.  It may be years before a company shares your info or has a data breach, and you may well have forgotten that one time you registered an account at chimptrek.com so you could post your favorite Freddie-Luc Picard quotes.   What?  It’s just an example, I swear!

We couldn’t agree more.

Sign up for The Decemberists’ newsletter here.

Related: Warning Signs That Your Client Is Spamming

Hi MailChimp,

My name is [NAME] and I am the Marketing Director of a Group Buying site in [COUNTRY] ([COMPANYNAME]).

I am currently looking to upgrade our Email Marketing System and am in conversations with [COMPETING ESP]. I have never used [COMPETING ESP] and have used MailChimp.

[COMPETING ESP] seem to think that they can gaurantee 20% better deliverability than MailChimp. Is that something you can disprove? How can you disprove this (e.g. comparison of technology, types of examples)

We will be sending many emails by the end of the year (in excess of 200 million). Can you guys effectively handle this type of volume?

This is the response that I proposed:

We published this page over here to show that so long as you, the sender, follow best practices, MailChimp’s infrastructure can help you achieve 99% (and sometimes 100%) delivery to the inbox. But if [COMPETING ESP] can actually get you 120% to the inbox, then wow. By all means, you should use them.

and that is why I am not on the front lines answering questions anymore. In fact, I’m not sure how I’m still allowed to answer comments on this blog.

Fortunately, our support team has a better, more professional response to this question (hyperlinks and emphasis below are added by me):

Hello [NAME],

Any email marketer can technically obtain 100% delivery so long as the following is in place:

• Data is collected through confirmable opt-in methods.  There are no assumptions about permission in play. (The account owners responsibility)
• Content sent is 100% spam filter safe. (The account owners responsibility, which we can help with this and this)
• Your delivery solution dynamically monitors and polices its environment for misuse and aggressively closes accounts that create situations that could harm other accounts within the system. (The ESP’s responsibility)

This is the technology that we use to police our service for bad apples:

http://www.mailchimp.com/omnivore/ (an overview)

http://blog.mailchimp.com/project-omnivore-declassified/ (more technical background)

You might want to review the account shut down stats for [COMPETING ESP] for insight into their protection measures.

This page provides additional deliverability information with regards to our service (including our definition of the word, and the seed list we use):

http://mailchimp.com/about/deliverability/

The delivery graphs there give examples of how someone that moves to our service, and who follow best practices outlined above, can quickly obtain 100% deliverability. There is a sample group buying site there, which you might find useful, as well as this article about daily senders, by someone on our delivery team.

Our system currently sends on average over 1 billion emails a month. So 200 million annually is not at all a problem.

I think, in a nutshell, the answer is that “Most well-established ESPs have gotten their infrastructure set up and proactively monitored to allow for extremely good delivery to the inbox. The rest is up to the sender. And if the sender does screw something up, it’s the ESPs responsibility to purge them from the system with extreme prejudice (whenever gentle hints and educational intervention are not enough).”

Other deliverability related stuff you may be interested in:

• How To Avoid Spam Filters (free guide)
• Email Delivery for Geeks (free guide)
• Warning Signs That Your Client is Spamming (free guide)
• Priority Inboxes and Measuring Engagement
• Daily Senders and Deliverability

If you’re interested in anti-spam topics, and you enjoy reading long technical documents composed in the courier font inside very tight margins (like I do), be sure to read this article about a best practices document that some blocklist operators have been working on since 2004:

“In any case, for those who choose to slog through it, BCP 07 offers insight into the way the major blacklist operators think”

I don’t handle our abuse or deliverability stuff anymore at MailChimp (much smarter people have taken over) but I remember the early days, when the behavior of just one MailChimp user would get our entire IP range blocked somewhere, and we’d have to jump into public forums and beg forgiveness (then endure all the ridicule) before getting delisted. It’s matured so much since then.

I also remember dealing with blocklists that banned MailChimp, but they were mysteriously also blocking the entire Internet too. So I found this interesting:

“A number of DNSBLs have shut down operations in such a way as to list the entire Internet, sometimes without warning.  These were usually done this way to force DNSBL users (mail administrators) to adjust their DNSBL client configurations to omit the now inoperative DNSBL and to shed the DNS query load from the registered domain name servers for the DNSBL. “

There are also guidelines in the document for DNSBL users, which make my imagination run wild about what incidents prompted the need for them.

This one, for example:

The DNSBL user MUST ensure that they understand the intended use of the DNSBL.  For example, some IP address-based DNSBLs are appropriate only for assessment of the peer IP address of the machine connecting to the DNSBL user’s mail server, and not other IP addresses appearing in an email (such as header Received lines or web links), or IRCconnections etc.  While a DNSBL user may choose to ignore the intent of the DNSBL, they SHOULD implement any variance in compliance with the DNSBL usage instructions.

For example, one of the requirements of some DNSBLs is that if the DNSBL is used contrary to the usage instructions, then the DNSBL user should not identify the DNSBL being used.  Furthermore, it is the DNSBL user’s responsibility to mitigate the effect of the listing locally.

Any system manager that uses DNSBLs is entrusting part of his or her server management to the parties that run the lists.  A DNSBL manager that decided to list 0/0 (which has actually happened) could cause every server that uses the DNSBL to reject all mail.

• Remember that blog network that just got hacked, and how all their user data was posted to the public? Wonder if any bad guys are importing that email list into MailChimp anywhere. Would be nice to shut them down, and maybe even report them to the FBI.
• Hey, what if we purchased some spam lists ourselves, and just used them to scan all users’ imported lists for high levels of correlation?
• Across all the emails we’ve ever sent, what’s a realistic “average shelf life” for a subscriber’s engagement?
• Is there a *real* “best time” and “worst time” to send email? Of course people will always say “it depends” but what if we actually crunched (all) the numbers anyway? Would we find interesting patterns?

And some questions can be real dilemmas, like:

• If user X imports a list, and we find a bunch of hard bounces, why don’t we prevent those bad email addresses from being imported into our system by user Y? (after all, lots of bounces can lead to delivery problems at some of the big ISPs)
• If we know a particular subscriber is a habitual (false) complainer, should we keep allowing them to subscribe to lists that we host? Even if there’s double opt-in proof?

MailChimp Engineers: “Shutup, already. Go look it up yourself.”

I guess all these questions finally annoyed our engineers enough to make them setup The Email Genome Project, which scans MailChimp’s 600,000 users, the hundreds of millions of subscribers they manage, and the 40 million (and growing) messages they send every day for nuggets of information that we can use to improve our deliverability and train our Omnivore abuse prevention algorithms.
The fun part of all this? The nerds get to play with cool toys…

First, they setup a server that’s used for some occasional pre-test “heavy lifting.” To be honest with you, I don’t think they really needed this one. I’m pretty sure they got it for fun. Whatever the case, here are the specs:

• 4 x Xeon X7550 CPUs, each 8 cores @2.0Ghz with HT
• 128 GB of DDR3 RAM
• Hardware BBU-backed raid 10 of Intel X25-E SLC SSDs

And then they setup another server that is not quite as impressive (with “only” 2×6 core xeons for a total of 24 threads, 36 GB RAM). This one was configured more for storage, with a 12 disk raid 10 of 15k SAS drives with ~4TB of usable raid 10 space.

I pretty much have no idea what I just typed there. Sounds impressive, though. The monthly bill certainly made an impression on me.

But hey, all in the name of R&D. If they wanna use the toys to play Doom (people still play that game, right?) or test their password cracking skills, it’s all good.

Anyway, the high level goal of the Email Genome Project is to help improve the email ecosystem. Specifically, we want to provide answers — fast. The more we learn about email, the better we can help prevent the abuse of it.

We’ll talk more about our findings here on the MailChimp blog soon.

For now, to get a feel for what kind of data our Email Genome Project can produce, you should sign up to Dan Zarrella’s “Science of Email Marketing” webinar.

He asked us a few questions about email marketing. We scanned 10 billion emails, and gave him some answers:

http://www.hubspot.com/the-science-of-email-marketing/

The experiment only involved sending roughly 7,000 email campaigns over to be reviewed. But within the first 2 days, we started getting back some unexpected, yet fascinating results.

In particular, there were certain email templates that kept getting repeatedly flagged as spam by these human reviewers, even though they weren’t spam at all.

All these “false positives” had some common design traits, so we thought we should share our findings…

How Did The Experiment Work?

When Omnivore detected an email that had traits of potential abuse, we sent it to Mechanical Turk. A copy of the email (sans private data, like recipient information) was displayed inside of an interface that looked something like this:

In general, we listed some rules at the top, then presented the campaign below it, then asked the reviewer to tell us if the email violated any of the listed rules, back at the top of the page. User Interface snobs will notice that in general, this interface looks like it was QWERTY-fied (designed to slow users down a little). We could’ve used very simple “Is this spam? Yes/No” buttons, but you don’t want people judging too fast.

How Effective Was The Experiment?

The experiment went as well as you’d expect, using people who weren’t heavily trained on the intricacies of permission-based email marketing. Generally speaking, Turkers like to work fast, so they’re best for picking out the most egregious offenders (think along the lines of porno or pharma spam). To that end, they’re great at catching the really evil spammers who try to penetrate into our system and send extremely bad stuff that would jeopardize our deliverability.

But when it came to reviewing an email from, say, a reputable business that purchased a not-so-reputable list from a local chamber of commerce, the reviewers experienced some difficulty. So crowdsourcing is good, but not a silver bullet with respect to abuse prevention (we are still crowdsourcing, but the experiment has changed significantly).

Though we weren’t thrilled with the initial results, this exercise revealed a lot about how people look at email design.

21 Seconds To Decide

Mechanical Turk measures how much time people spend performing each review, so we can tell when people are just clicking random stuff and moving on to their next task. On average, the human reviewers spent only 21 seconds reviewing these “false positive” emails. Now, we can’t read their minds, so there’s no reliable way of telling if they bothered to check for “permission reminders” or “CAN-SPAM compliance” in the footers. But it’s safe to say they weren’t doing a very thorough analysis. I’d wager that most of that 21 seconds was spent reading the criteria at the top of the interface, and not the email itself. They definitely weren’t visiting the senders’ websites to see if there was a proper signup form, and testing to see if they used opt-in best practices. They were making relatively quick, gut-level decisions on whether or not an email “looked spammy.”

The False Positives

Below are some email designs that kept getting marked as spam by Mechanical Turk reviewers. Keep in mind that at the time of this experiment, none of the senders of these emails were determined to be abusive. Their email stats suggested they were sending permission-based emails. Their recipients probably knew the emails were legit — but our independent reviewers did not.

1. Want to learn Photoshop?

In general, I think the above email has got some layout issues that make it look a bit sloppy. Their images are breaking the template. At the top, where people are accustomed to seeing a logo, the sender only used text. In fact, the text isn’t even the company’s name, but a bright red “salesy” kind of question: “Want to learn Japanese or Chinese?” Doesn’t exactly inspire confidence that you know your recipient, or what he’s interested in. Unfortunately, the Chinese characters don’t help their reputation much either. We’ve all received a bit too much of this in our inbox:

2. The Red Flyer

I’m sure that loyal customers of this local pizzeria were happy to get an offer for a free t-shirt:

But I don’t think our human reviewers liked the “hyperlink blue” verdana font, then the giant red “FREE” text below that (then the green text below that, then the blue text below that, then the gray text below that). Something about this email made it look more like a stock template for a flyer, not an email newsletter to loyal customers. I couldn’t help but think that the scrunched up airplane logo looked like those images that spammers try to skew, in order to get around anti-spam filters who scan the content of images:

Aside from the image quality issues, some extra copy could’ve been added to demonstrate that this email was being sent to their customers. Don’t get me wrong. T-shirt giveaways can be  extremely effective (here are some stats to prove it), but you should probably do more than just yell “FREE T-SHIRT!”

At the very least, an image of the actual t-shirt seems in order.

Here’s a nice example from ScoutMob:

3. Not Plain Enough Text

This email repeatedly got marked as spam by our reviewers:

You’ll notice it has no images. No branding, no logos, no photos.

Yes, one could make the case that plain, old-fashioned, text-only emails can be more personal, and therefore more effective under some circumstances.

But if you’re gonna go all-text, you need to go all the way, baby. Centered text, colored backgrounds, and colored borders look like you’re going for an HTML email look. But when you fail to include any logos or images, it looks half-baked. Like a spammer, getting all “Rich Text:”

Even if you don’t have a logo, one way of showing your brand is to include your website’s domain. But this sender used the bit.ly URL shortener instead:

In their defense, that’s probably because the link to the event they’re promoting was really long or something (webinar links get that way sometimes). The problem is that spammers are known to hide malicious links behind reputable URL shorteners (see: URL Shorteners and Blacklists), so that helpful little link just ends up hurting them.

4. Read it and Weep

This one was actually surprising to me, because I thought it was well designed:

The title font even looks customized (it’s not arial, it’s not verdana, and it’s certainly not comic sans). It’s laid out pretty nicely. The pink is a custom color, too. The only possible problem that I can see is that it’s extremely text-heavy, with zero images. To the untrained eye, it almost falls into that “not plain-enough text” category above, but this doesn’t look half-baked or sloppy at all to me. This email shows signs of actual craftsmanship and skill with typography (web design is 95% typography, right?). This sender’s subscribers are probably fine with all this text (the sender is an author, after all). But to our independent reviewers, this email apparently looked pretty spammy. In this case, I personally wouldn’t change my design or behavior. If I had to make recommendations, I’d consider adding elements that made it look more “newslettery.” Perhaps a small avatar of the author could be worked into the template’s footer, or some “share this on social sites” icons. If this is all about the written word, and images are forbidden, text can be ornamental too.

5. Set it and Forgot it

Senders that used one of our stock RSS-to-email templates seemed to get flagged the most:

As I write this article, we’re actually working on tweaking this template so that the header is more customizable (forcing the title to be ALL CAPS, in retrospect, was not a great idea).

But many of the bloggers who used this template didn’t bother customizing the RSS merge tags any further to include images from their posts. They didn’t customize the fonts, link colors, or anything at all, it seems.

I also wonder if, in some cases, the Table of Contents was so large, our independent reviewers didn’t bother scrolling down to look for real content. All they saw was a bunch of nonsensical looking TOC links. This happens if you update your blog frequently, but you schedule your RSS-to-email campaign to go out in weekly or monthly batches. Not that I’d change my behavior just for random Mechanical Turk reviewers. What your subscribers want is more important.

But there’s a broader lesson here on image vs. text balance. A similar example plucked from my spam folder in Gmail:

Why this is important to email marketers

When you send a lot of email marketing, even to a totally permission-based double opt-in list, you’re going to get some spam complaints from your recipients. It’s inevitable. Sometimes, it’s because they’re too lazy to click your unsub link, they think the “spam” button is the unsub link, or sometimes it’s because they forgot signing up to your list (maybe because you send infrequently, like me).

And sometimes, when your email is marked as spam, a human from an ISP, or a human from an anti-spam organization, will actually do a manual review of your email (See: “Who’s secretly reading your emails?”). Some anti-spam organizations use volunteers, who are driven by passion more than pay (nothing wrong with that, but you have to wonder how detailed their training is). We’ve experienced enough “your client’s email has been reviewed by our team, and determined to be spam, so we’re blocking your IP range” situations to know that those reviewers don’t always do a thorough analysis of your list management practices (not part of their job description anyway). This is partly why our own terms of use seem so strict to some. ISPs get complaints, they look at your email, and they make a split-second decision to “blacklist or not.”

So even if you do your list management right, and you design everything perfectly around your subscribers’ expectations, we always recommend that you give some consideration to this “secret” audience that also reads your email (See: “What makes a good permission reminder?”). Don’t bend over backwards for them, or anything.

It’s kind of like how your mother always told you to wear clean underwear, “in case you’re in an accident.” Take a good look at your email templates, and ask yourself, “If my email got reported as spam, and some spamcop laid his eyes on it, what would they think? Would mom be proud?”

Related:

• How your email design can get you blacklisted
• Stupid Email Design Mistakes
• How to avoid spam filters (the non-human kind)
• Want 700,000 HTML email templates? (more fun w/Mechanical Turk)
• Is your email marketing human?

We also discussed how MailChimp is using engagement to decide which emails to clean from our senders’ lists.

Yesterday, we learned that Hotmail is also tracking user interaction within their inbox in order to determine which emails to throw away, and which to keep:

Hotmail Using New Metrics to Consider Inbox Placement by George Bilbrey on MediaPost

Reading that article from George made me realize I’m way, waaaaay behind on my Hotmail knowledge. I haven’t checked out their interface for ages, except for when I glance at our Inbox Inspection reports (and skimming the occasional article about their “sweep” feature).

I guess it’s because my domain performance stats on all my campaign reports have been showing this trend over the years:

But they’re still HUGE. So I setup a Hotmail account today, and at first glance, it’s kind of neat. I like that they have smart folders for shipping notifications, photo storage, and Office Docs (makes sense). They say something about organizing your social network notifications too, though I haven’t found that in my first few minutes of testing here.

If you haven’t checked out Hotmail in a while, you can catch up on their new features at the Windows Live blog.

Competition between Microsoft and Google is good for all of us. I couldn’t help but notice Microsoft’s video, and compare it with Gmail’s. Wondering which one our readers like better:

Microsoft’s video:

Gmail’s video:

It would be all fine and dandy if people would follow delivery status notification best practices and guidelines. But they don’t. Sometimes this is a reaction to spam, and sometimes it’s just ignorance.

For example, some server admins insert snarky messages in their email headers, like “We don’t want your message. If you send email to us again, we’ll report you.” Well, that’s their prerogative and all, and we’re happy to never send to them again, but if they simply hard bounced the email, we’d be able to clean it from the list faster.

Then there are some ISPs who are downright deceptive with their bounceback codes…

They’ll send back a bounce that tells us that the intended recipient doesn’t exist. But look closely at their bounce headers, and you see little messages like, “but if you wait a few hours and try again, it’ll get through — wink wink.” An interesting way to tell if there are humans sending the email.

Then there’s the problem of “silent dropping:”

“As discussed in Section 7.8 and Section 7.9 below, dropping mail without notification of the sender is permitted in practice. However, it is extremely dangerous and violates a long tradition and community expectations that mail is either delivered or returned. If silent message-dropping is misused, it could easily undermine confidence in the reliability of the Internet’s mail systems. So silent dropping of messages should be considered only in those cases where there is very high confidence that the messages are seriously fraudulent or otherwise inappropriate.” Source: wikipedia

And there have been cases where an ISP will temporarily go down for hours (or days), and in the meantime, they send you back hard bounces or erroneous “you’ve been blocked” reports. Should you clean those hard bounces from your list? Technically, it’s a “best practice.” But clearly, the ISP was broken when you sent. Hardware just breaks sometimes (See: Not All Delivery Problems are Spam Related).

Some receiving servers have sent back hard bounced messages that were intended for recipients that we know exist, because we have double opt-in evidence, and open/click actiivtiy. We find out about these problems when recipients complain to the sender about not getting the email they requested, and the sender escalates it to us, and then we trace it back to the recipient’s IT guy setting up “custom” rules. To be clear, it’s their prerogative to setup their custom rules. We don’t hold it against them (spam’s ruining it for everyone). But this does create a problem that requires a custom solution of our own.

Good Deliverability Depends on Proper Bounce Management

See why bounce cleaning can be frustrating? No wonder people who try to manage their email marketing in-house see such dramatic improvements in deliverability when they switch to an ESP (case study). They’re usually unable to properly clean the bounces from their lists.

So they don’t.

And if you keep sending messages to non-existent accounts, ISPs will block you because you look like a spammer who purchased an old email list.

Improper bounce cleaning can seriously damage your domain reputation.

Also, we’re seeing new trends in the way our customers send emails. People are automating more with RSS-to-email, and via our API. More daily senders with extremely large lists (daily deals, mobile apps, location-based check-in services, etc) are depending on us to get their emails delivered, but also depend on us to intelligently manage those lists. Simplistic bounce cleaning rules, combined with deceptive bounce errors, can result in their lists shrinking faster than new members can opt-in. This, in turn, often results in irrational behavior by the sender (purchasing lists, using bad/old lists, un-bouncing everybody, ESP-hopping with old, uncleaned lists, and on and on).

So we’re tweaking the way we handle bounces.

Our strategy for a long time now has been to perform deep, ongoing analysis of bounce headers in order to create “the most insanely thorough bounce back interpreter holy-grail known to man” (and we usually end that statement with an evil, nerdy laugh). And we’ve come a long way with that approach.

Moving forward though, MailChimp will be factoring engagement activity into our bounce cleaning decisions (read about how MailChimp measures engagement).

For example, if we send an email and a receiving server tells us that a recipient “does not exist,” but we have open and click activity in the last 45 days to prove otherwise, we’re not going to blindly clean that recipient from the list. We know they exist, and we know their account works, so we’re going to give them a few more chances than we normally do. If, however, we see that there’s very little (or no) activity by that recipient, we clean them under the same rules we’ve used in the past.

We’re not going to get into specifics about how many stars justifies a “clean vs. a keep,” or exactly how many chances we give hard and soft bounces. The algorithm will surely be adjusted and tweaked over time. The point we’re trying to make is that email is evolving faster than ever (thanks to changing social and mobile behaviors of recipients and senders), and MailChimp is adapting and innovating along with it. Even in the very un-sexy area of bounce management.

For those of you who don’t know, we built Omnivore in order to prepare for our big Freemium plan that we launched back on September 1st, 2009. We didn’t want to offer a free email marketing service without having a scalable system in place to protect our deliverability (not to mention the sanity of our Compliance Team).  Good thing, too.

In just under a year, MailChimp grew from 85,000 users to over 430,000. We couldn’t have grown 5-fold like that without Omnivore.

Here’s an update on what we’ve learned so far…

Since September 1st 2009 Omnivore has:

• Issued 69,927 warnings to 24,119 users for exhibiting bad behavior. Warnings like, “Hey, we detected a lot of unsubscribes from that last campaign — if it continues, we’re going to have a deliverability problem.” Another warning example would be something like, “Whoah, that last campaign had a higher than normal bounce rate. Something’s going on with your list hygiene. Here are some tips for you to address that issue before it gets worse.” Warnings usually never need a reply. They’re simple observations that let you know something’s wrong, and if it’s not corrected, could lead to more issues, which could lead to suspension.
• Automatically suspended 8,770 users. This typically happens when Omnivore sees something really alarming, and just can’t allow an email to leave our system. When a user’s account gets suspended, it’s placed into a queue for human review. Our compliance team basically investigates to see if it was a false positive, sends tips to users if it was an innocent mistake, or in some cases, might decide to permanently shut down the sender.
• Of those suspended accounts, 1,879 ultimately had to be shut down. Shutdowns don’t always mean the sender was evil. Sometimes they just mean that a sender might be sending content that’s too risky, and receives more than the normal amount of delivery problems or abuse reports. Even if they’re totally innocent, they can still cause harm, and we have to shut them down.

When I first wrote about Omnivore, I was very careful to explain that it was new, and still had a lot to learn.

Over the last few months, it’s learned a lot.

New Shades of Gray

Our approach with Omnivore has been primarily to look for those things that spam filters don’t. For example, we could simply scan outgoing email with Spam Assassin’s criteria, and block offending messages. But that would only catch the “black and white” stuff. That’s fine for catching the horrible appendage-enlargement spam we’re all so familiar with. But ESPs deal with “ignorant spam” more than “evil spam.” Ignorant spam is harder to define. It’s a gray area.

Speaking of shades of gray:

And that’s the stuff Omnivore looks for.  Stuff that looks like perfectly legitimate business mail, and that would slip past most spam filters, but then generate a ton of spam complaints from recipients (traits that humans think are spammy, but that spam filters miss).

Since launching, we’ve discovered even more shades of gray in the abuse spectrum.

Lots more.

Investing in the ecosystem

And we’ve built new tools to detect those shades of gray. I won’t divulge our entire budget for the Omnivore program, but I can tell you that we’re investing $20,000 per month on monitoring just one of those new “shades of gray.” Not to mention our investment in new staff, and in training. We are committed to protecting the email ecosystem. That’s not to say that our colleagues, like ConstantContact, ExactTarget, and other major ESPs, are not. They all devote a tremendous  amount of time, energy, money and resources on this stuff, and we’d be remiss not to mention them. Especially since they’re so willing to share their research with each other. Without a properly functioning ecosystem, we’re all dead.

We just want our customers to know how much thought goes into abuse prevention. It’s important to convey that.

For example, if we catch a spammer trying to hack away at our system, we almost always trace them back to some small, free ISP that they’re using to host all their malware. What do those ISPs usually tell us? “Abuse is inevitable and a part of life, and we’re sorry, and the account’s been shut down. Goodbye.” Hmm. We can’t help but wonder if they’re doing much to prevent that abuse in the first place. We’re sure they are, and we’re sure they’re being terse for legal reasons. But we still wonder.

We don’t want our customers to wonder.

New “Three Strikes” Policy

We don’t think Omnivore is perfect. But we’re much more confident in its ability to detect and prevent abuse now. So we’ll soon be implementing a new policy. If any sender on MailChimp is suspended three times in 6 months (whether the suspensions are a result of bad behavior or innocent mistakes, and whether the suspension was initiated by Omnivore or staff), Omnivore will permanently shut down the account. As I explained above, suspension isn’t always because of evil. Often, the sender made a totally innocent mistake. And after each suspension, our team always sends helpful recommendations to get senders back on the right track. We’ll even point some of them to 3rd party deliverability specialists, who can train them on best practices. So there’s rarely a valid reason for having 3 suspensions inside a 6-month period.

Related:

• Project Omnivore Declassified
• Spam lawsuits – What’s the worst that can happen?
• Preventing False Abuse Complaints