Nov 18, 2014

Perfecting Our Own Automation Recipes

For years, MailChimp delivered our Getting Started series to new subscribers with autoresponders. But when we completely rebuilt our automated emails, turning them into a more robust and flexible feature called automation, it was time to—as my dad still likes to say—kick it up a notch. We took a hard, close look at what, when, and to whom we were sending, did some testing (of course), and came out the other side with improved open and click rates. Here’s how we did it.

Segmentation

Initially, we messed up by serving all different types of customers the same way. Since day one, everyone who signed up for the series received the same campaigns in the same order. Are you moving a million addresses from another email service provider? Let’s start with templates. You’ve got no list and no subscribers? Here’s a few templates. Oh, you’re sorting daily sends to different groups? Templates. Prepping your first ever newsletter? Yep, you guessed it.

It took us a while to realize that this maybe wasn’t the best approach. Eventually, we learned that if we wanted to turn new customers into MailChimp pros, we had to consider where they were starting from.

Signup questions

When our research team added a few questions for new signups, we knew this data could help us tailor our content. Pete, our lead engineer, connected the responses through the MailChimp API and added them to groups in our list.

Inspired by the return of Twin Peaks, the series now looks like this:

Getting Started map

We built our own automation rules to create two tracks. So now, when new users sign up for the series, they enter a specific workflow determined by the age of your company and whether or not you’re setting up MailChimp for a client.

Timing

We started testing send times toward the end of our autoresponder era. At the time, each email was spaced out by 4 days. But backed by our study on subscriber engagement, we sped up the series and followed each trial’s results for a week.

Email Delay Open Rate Click Rate
Hourly, 4 days, 7 days, 10 days 41.7% 4.2%
Hourly, 3 days, 6 days, 9 days 42.2% 4.3%
Hourly, 2 days, 4 days, 6 days 42.5% 4.4%
Hourly, 1 days, 2 days, 3 days 42.4% 4.3%
Hourly, 2 days, 4 days, 6 days 42.6% 4.5%
Immediately, 2 days, 4 days, 6 days 43.5% 5.9%

For 3 weeks, engagement climbed. Finally, in the fourth week (when we were sending 4 emails in 4 days), it dropped off. Reverting to the previous step, the pace picked up again. Just like there’s no perfect second to send a single campaign, there’s definitely no catch-all solution for automating. But testing can definitely get you closer!

The biggest bump came after we converted to automation. Using autoresponders, the initial email was limited to the top of each hour. Using workflows, we can send immediately. Now, when a new user logs in for the first time, Getting Started is already in her inbox. With this one change, opens and clicks increased by 6% and 17%, respectively.

Content

We designed the new Getting Started series to be flexible. Our autoresponders used to be long, dense, and fixed. But with workflows, we can add new emails or adjust timing with just a few clicks.

And it’s easy to trigger new content when someone’s ready. If a new user wants to collect subscribers on Facebook, we can send a follow-up that explains Facebook Custom Audiences. If they’re a small- or medium-sized company that’s clicked over to eCommerce360, we can share information on MailChimp Snap.

Lately, we’ve split our series further to test specific tips. Instead of a campaign about design, we sent focused tips on segmentation and creating an A/B test, and followed the actions in each user’s account. Automation lets us make quick changes and adjust our content on the fly.

The Future

MailChimp automation keeps getting better, so there’s still more to try. We can restrict delivery to certain hours or follow subscribers with Goal or even integrate our transactional emails. Some things will work and others might not, but it’s always fun to perfect our recipe. Bam!