Oct 3, 2012

Three Billion Emails

We sent 3 billion emails last month, and that’s way cool! There was a time (and I’m talking about last year) when the idea of sending a billion emails a month blew my mind. Now I take it for granted that MailChimp can handle whatever we throw at it.

I thought I’d share some of my enthusiasm, so if you look below you’ll see a chart with our monthly volume since 2011. I also included a list of our peak performance stats, like our best second, minute, day, etc.

Joe gave you an idea of what MailChimp’s growth looks like from a data perspective, but there’s a whole lot we do on the delivery side to complement that volume and make sure your emails deliver as quickly as possible. As more and more emails go out each month, we have to think beyond simply increasing our capacity. We have to get nerdy.

Nerd out with me for a minute.

Peak Performance Stats

  • 6,700 emails per second
  • 283,000 emails per minute
  • 12,000,000 emails per hour
  • 151,000,000 emails per day
  • 755,000,000 emails per week
  • 3,000,000,000 emails per month!

IP + Volume = Reputation

You may not know this, but we can’t take a new IP and push half a million emails over it all at once. The problem is that cold IPs don’t have any reputation. Email receivers like Gmail and Hotmail would probably bounce all that traffic if we tried. It’s no joke.

Instead, IPs have to be slowly warmed up. We like to take 6 months to completely warm up a cold IP. Our automated systems make sure these IPs send a little more traffic day by day for 180 days, and we monitor their delivery the entire way to make sure things are going smoothly. What I’m trying to say is, it’s not easy to add more capacity.

Back in 2011, I used to worry about large senders. What if someone signed up to send a ton of emails right away and we didn’t have enough IPs. At the time, we were using maybe 300 IPs to send all of our traffic. A really large sender can monopolize 3 or 4 or 20 IPs all by themselves. A handful of large senders could easily tie up a significant portion of our capacity, and that would slow down sending for everyone. Scary.

Right now we’re using about 1,000 IPs to send all of our traffic, but we have a lot more users than we did a year ago. We have a lot more large users for sure. Why am I not worried? One word: drasticallyoveranticipating. Okay three words: drastically over-anticipating. Wait, that’s two words? ← Internet, I have a grammar question. Um, anyway…

We are always warming up new IPs. Having a reserve of IPs on deck keeps us really flexible, and that’s what puts my mind at ease. Whether it’s a large user coming in or a particularly large Cyber Monday, we have extra capacity being created automatically.

Large Sender + Campaign = Solution

Every now and then, a new user comes along and uploads the biggest list we’ve ever seen. It was really exciting the first few times because there’s so much we can learn from large senders. You’d be surprised at how sending a campaign of 10 or 20 million emails is different than sending a million smaller campaigns.

One of the first things we decided to do was to break up your campaign payload into chunks. Payload? Chunks? What? When you send a campaign, MailChimp spins up a file that contains all of your content and all of the subscribers you want to send to. That’s a payload file.

The problem is, once a payload file reaches a certain size, it takes longer and longer to add the next line (i.e. the next subscriber). For super large campaigns, it can end up taking an hour or more to build that payload. Ugh! When you hit “send,” you expect your emails to go out immediately, right? By dividing up our payload processing into smaller chunks, we make sure that large campaigns start going out right away.

We originally made that particular change because a single user was trying to send a campaign to a million subscribers. Chunking payloads helped them a lot, but it also helped all MailChimp users get their campaigns out faster. We love it when that happens, and it happens all the time. Like I said, there’s a lot to learn from large senders.

Knowledge + Email = Delivery

So I mentioned Cyber Monday a while ago. Understanding cycles and trends is a big deal for delivering email. If you want to over-anticipate, you have to anticipate.

For instance, we’re currently preparing for December because we know it’s going to be intense. Throw in Valentines Day, Mother’s Day, and the 4th of July, and there’s always some big holiday on the horizon. But holidays are known quantities, and there are other kinds of trends we have to be aware of.

When Gmail launched Priority Inbox, all of our delivery settings to Gmail had to change overnight. The speed that we send emails, the number of simultaneous connections we make, the number of messages per connection – all of it had to change. The overall effect was that Gmail started receiving email at a slower rate. With some quick testing and a lot of experience, we were able to keep emails flowing.

By the way, we also took note when Yahoo announced they were moving to engagement based filtering. We didn’t have to make any changes for that one, but engagement is a big deal and Van has some great tips.

MailChimp + You = 3,000,000,000 Emails

I guess what I’m trying to say is, there’s a lot of thought and nerdiness that goes into sending 3 billion emails a month. As I’m finishing this blog post up and getting my charts made, I just realized we sent another 3 billion emails in September. I keep saying “we” sent 3 billion emails, but I should be saying you, as MailChimp users, sent 3 billion emails. You rock!