As you can imagine, the MailChimp Abuse Desk receives some really nasty emails from people. Fortunately, we also get a lot of very positive emails from people trying to do the right thing, and who genuinely appreciate the measures we’ve put in place to protect the email ecosystem. If you work in an abuse desk somewhere — either an ISP or an ESP — this post is for you.
Here’s a nice email we got today:
Some interesting stuff from Ken Magill.
If you’re interested in anti-spam topics, and you enjoy reading long technical documents composed in the courier font inside very tight margins (like I do), be sure to read this article about a best practices document that some blocklist operators have been working on since 2004:
“In any case, for those who choose to slog through it, BCP 07 offers insight into the way the major blacklist operators think”
I don’t handle our abuse or deliverability stuff anymore at MailChimp (much smarter people have taken over) but I remember the early days, when the behavior of just one MailChimp user would get our entire IP range blocked somewhere, and we’d have to jump into public forums and beg forgiveness (then endure all the ridicule) before getting delisted. It’s matured so much since then.
I also remember dealing with blocklists that banned MailChimp, but they were mysteriously also blocking the entire Internet too. So I found this interesting:
A screenshot from an Elf Yourself clip being passed around our office:
What I find interesting is not so much how someone at MailChimp put my face on one of the dancing elves (though I did get a chuckle out of my breakdance routine), but that this year, there’s a “Report for Abuse” link in the upper left corner.
We recently experimented with crowdsourcing the review of outgoing campaigns from MailChimp’s servers. Normally, if our Omnivore algorithms detect something suspicious about a campaign, we’ll automatically suspend the account and follow up with a review by our internal Compliance Team. But we’ve been testing the idea of also sending the campaign to Amazon’s Mechanical Turk service for manual review by humans. We simply showed the email to a “turker” and asked them, “Is this spam?”
The experiment only involved sending roughly 7,000 email campaigns over to be reviewed. But within the first 2 days, we started getting back some unexpected, yet fascinating results.
In particular, there were certain email templates that kept getting repeatedly flagged as spam by these human reviewers, even though they weren’t spam at all.
All these “false positives” had some common design traits, so we thought we should share our findings…
In January, we announced Omnivore, our massive anti-spam research project that ran 61 trillion email data comparisons using genetic optimization algorithms in order to teach our network how to automatically detect and prevent abuse.
For those of you who don’t know, we built Omnivore in order to prepare for our big Freemium plan that we launched back on September 1st, 2009. We didn’t want to offer a free email marketing service without having a scalable system in place to protect our deliverability (not to mention the sanity of our Compliance Team). Good thing, too.
In just under a year, MailChimp grew from 85,000 users to over 430,000. We couldn’t have grown 5-fold like that without Omnivore.
Here’s an update on what we’ve learned so far…