Sep 22, 2015

Compliance Insights for MailChimp Pro

Like all the other departments at MailChimp, the primary goal of the Compliance team is to empower users. We’re always looking for ways to do this better, and working on MailChimp Pro, our brand new feature set for quickly growing businesses with lists of 50,000 or more, offered tons of opportunities. So while Pro was incubating, a few of us got together to figure out how we could provide users with better insights into our process.

Some folks might think Compliance exists only to find bad users and shut them down, but there’s more to it than that. We know good senders occasionally mess up, and we also know that getting shut down for a single mistake can materially impact a user’s business. A lot of folks worry, “Am I OK, or am I doing something wrong?”

We built Compliance Insights to help MailChimp Pro users answer that question for themselves. We believe our customers should have the ability to clearly understand their compliance risks and the data they need to avoid abuse incidents whenever possible.

To make Compliance Insights, we knew we’d have to expose some of the inner workings of our compliance process. Namely, that of Omnivore, the automated part of our abuse-prevention initiative that scans data across our entire system and flags accounts for the folks on our Compliance team to review. Here’s how we did it.

Tracking your current reputation

When you’re sending email, reputation is everything. You have a reputation with your subscribers, your From address has a reputation, the IPs you send through have reputations, and every ISP you send to decides for themselves what these reputations are. Even Omnivore has a concept of what each user’s reputation is with MailChimp.

Before, only our Compliance team had access to Omnivore’s reputation info. But now, MailChimp Pro users can easily see this reputation for themselves:

chart_currentreputation

The new visual scale is built from two key elements in Omnivore’s data stream. First, Omnivore tracks your entire engagement history for every campaign you’ve ever sent. That’s the history of your opens, clicks, bounces, and abuse complaints. We weigh new campaigns more heavily than older campaigns to ensure we know how users are changing over time. Second, Omnivore tracks campaigns with particularly high bounce or abuse rates. We call a campaign like this a "negative send event," but because that’s an ominous and terrifying phrase, let’s use 📬💩 for the purposes of this blog post.

The math that combines our 2 data streams into a single value is fairly complex, but here’s the simplified version: your engagement history determines how high you can go, and every 📬💩 drags you down. If you look closely at the reputation scale, you’ll see a gray zone that reflects the reputation most MailChimp users have.

Since most users don’t have a 📬💩 in their recent history, you’ll likely only end up below this box if you’re a new user who hasn’t built up a campaign history yet (no big deal) or if you’re recovering from a campaign with particularly bad stats (slightly bigger deal). If you end up below the gray box because of the latter, you can use this tool to track how quickly you rebuild your reputation.

It’s worth noting that sending another 📬💩 while you’re below the gray box will lower your reputation even further. This can impact the severity of the response from Omnivore and our Compliance team. After each 📬💩, you’ll have to take specific actions in order to start rebuilding your reputation. These actions are designed to set you on the right path to building a healthy and engaged list.

Improving your future reputation

When building Compliance Insights, we wanted to do more than simply show users where they stand. With the entire engagement history of MailChimp’s 9 million users at its disposal, we realized Omnivore should be able to offer specific and actionable data about what items users can focus on to proactively prevent abuse issues and increase engagement.

Underneath the reputation scale, you’ll see status boxes for the 4 key engagement factors Omnivore tracks: opens, clicks, bounces, and abuse complaints. Your performance on these 4 factors directly affects how high your reputation can go. They also determine how quickly you rebuild reputation after a 📬💩.

chart_historicalreputation

Under each factor, we’ve included a link to a MailChimp Knowledge Base article that focuses on specific actions you can take to improve your campaign performance. We’ve found that users who aren’t following the best-practice recommendations in our KB articles tend to have a higher risk of sending campaigns with critically high bounce and/or abuse rates. If your historical bounce or abuse rates are listed as “poor,” Omnivore is trying to tell you that you have an increased risk of committing a 📬💩. By improving these factors, you can prevent a worse result down the road.

Success through empowered sending

Our compliance process isn’t always a fun topic, but it’s much better to learn about ahead of time than it is to play catch-up after Omnivore has flagged your account. Hopefully, tracking your MailChimp reputation and understanding the reasons behind it will set you up for success. If mistakes do happen, the data you’ll see in Compliance Insights is the same data our human review team sees behind the scenes. With great power comes great responsibility, so use this information wisely!