Nov 4, 2008

A/B Test Your Email Design and Content in MailChimp

Update (8/26/15): A lot has changed in 7 years! Check out our new and improved A/B Testing features, which let you test up to 3 variations on each campaign, using different content, send times, From names, and more.

Did you know you can use some nifty new advanced merge tags inside of MailChimp to test different designs or content in your email campaigns?

It’s a great way to test two different:

  • Incentives, discounts, compelling offers ("Free Shipping" vs. "15% off your order")
  • Headline font sizes, colors (do bright colors help?)
  • Headline word counts (long headline vs. short headlines)
  • Product images (standalone image vs. product in use)

to see which version gets more clicks and sales from your subscribers.

Plus, it’s super easy (no rocket scientists required). Here’s how to do it…

Just create an A/B campaign like you normally would. You might want to set MailChimp to not automagically send a campaign to the winner. Set it to 50/50 test groups, or set it to "decide manually." That way, you can look at your results and decide which campaign won yourself.

Then, in your email message, place this code wherever you want to test two blocks of content or designs against each other:

*|GROUP:A|*
This block of content/graphics will only show for group A recipients
*|END:GROUP|*
*|GROUP:B|*
This block of content/graphics will only show for group B recipients
*|END:GROUP|*

MailChimp will dynamically replace those merge tags with the proper content for each test group.

Members of test group A will see whatever you stick inside the *|GROUP:A|* tags, and members of group B will see whatever you stick inside the *|GROUP:B|* tag. Note that A and B groups are randomly and automatically chosen by MailChimp. That’s what’s so cool about our A/B tool.

Then, you can look at your campaign’s stats to see which content won.

Cool, huh?

And you can combine this with an A/B test of two different subject lines. Whoah. That’s like those fancy-schmancy multivariate tests the marketing nerds are always talking about.

For example, I sent an A/B test that showed group A got more opens, so that subject line won. But group B got more clicks, so that product graphic won. Next campaign, I’ll send it with subject line A, and product graphic B.

You can stick whatever you want inside those tags.

Text, links, pictures, tables. Even entire email template designs.

Yeah, you heard me right.

If you’re an advanced coder, you can switch to our advanced editor (where you copy-paste your own template code) and setup an A/B test of two totally different email designs. You’d do it like this:


<HTML>
<BODY>

*|GROUP:A|*
THE CODE FOR EMAIL TEMPLATE DESIGN A
*|END:GROUP|*
*|GROUP:B|*
THE CODE FOR EMAIL TEMPLATE DESIGN B
*|END:GROUP|*

</BODY>
</HTML>

Note how I took the HTML and BODY tags out of each of my different email templates, and then applied them around BOTH my A and B designs. I don’t think MailChimp cares about that so much, but I found it made things less confusing for MailChimp when you do a pop-up preview.

Give it a shot! But make sure you do plenty of testing before you send your campaign.

Click our "Send Test" button:

and send separate versions of A and B to yourself plenty of times to make sure you’ve got everything setup correctly. Seriously. You have to test your A/B tests, too.