Dec 18, 2009

A/B testing the word webinar vs. online training

We run a series of MailChimp webinars every week, so we’ve got links all over our website pointing over to them. Our website testing guru Stephen wanted to test the word "webinar" against phrases like "online training" and "live training" and "MailChimp training." He used Google’s Website Optimizer to run some A/B tests.

Turns out "online training" beats "webinar." By a lot:


Simply changing the word "webinar" to "online training" boosted click-throughs to our landing page 101%.

Then we got to thinking. We send email invitations to people who register for those webinars. Would we get better results if we switched the words in our subject line the same way?

Of course we couldn’t resist using our patent-pending email A/B tester.

We ran an A/B test on these two subject lines, sent to those who registered for the weekly webinar:


In our emails, the word "webinar" gets more opens than "online training."

Go figure.

No seriously, go figure this out please, because we have no idea why.

At first, we were bummed that results weren’t the same as the web page testing, so we did this for 2 weeks. What you see above is our 3rd A/B test. Every time, "webinar" wins.

Just goes to show. Better A/B test your assumptions.