Most people quickly scan the subject lines in their inbox before deciding which messages are worth their time and attention. With so much pressure on the subject line to entice the potential reader, we thought it would be interesting to see how much of a difference a single word can make in a campaign’s open rate.

To get some answers, we studied approximately 24 billion delivered emails with subject lines composed of approximately 22,000 distinct words. If you think that sounds like a lot of data, you’re right. We looked at subject lines both in general and within specific industries. Here’s a quick rundown of our criteria and approach:

1. Investigate campaigns sent by users from the United States with tracking turned on in the past year. Only consider campaigns that were sent to 500 recipients or more, and only consider campaigns sent by users who have sent 10 or more campaigns before.
2. For each campaign, calculate the open rate and standardize it using the user/list average open rate and standard deviation.
3. Remove special symbols and convert subject lines to lowercase. For any given word, average the performance of all related subject lines and perform t-tests to identify high impact words.
4. For every subject line being tested, create flags for the presence of high impact words. Perform a correlation analysis on word presence to determine which words are frequently used together. Create additional flags for frequently used word combinations.
5. Perform a linear regression analysis to estimate the impact each word has on standardized campaign open rates when accounting for all other tested words. Repeat this process on industry-specific data sets.

The numbers presented below are standard deviations from the mean open rate for a user/list. Words with positive impacts resulted in increased open rates, and words with negative impacts hurt those same rates.

Results for comparable word groups

The output of the regression analysis suggests that similar words often have similar impacts on open rates. Makes sense. Still, choosing the right words can result in higher open rates without altering the bottom line of your message. Again, to interpret these results, it’s important to know that a standard deviation is a standardized measurement of how much something deviates from the average value. One standard deviation for a user who tends to see large swings in open rates will be a higher percentage than it will be for someone with consistent open rates.  That means choosing words wisely will have a larger impact on open rates for people with a higher standard deviation, while users with very consistent open rates can expect to see smaller changes.

Personalization works

MailChimp’s merge tags let senders include first and last names provided by the recipients in campaign subjects or bodies. The impact this has on open rates has been debated before, but the consensus is that it’s positive. Our analysis found that personalization does indeed increase open rates. One of the most interesting findings is that, though the use of both first and last names in a subject is less common, it has the largest positive impact on open rates. Many of these campaigns seemed to contain highly personalized content.

Congratulations, *|FNAME|* *|LNAME|*

TED2014: Invitation to register for *|FNAME|* *|LNAME|*

Hi *|TITLE:FNAME|* *|TITLE:LNAME|*, please update your email preferences

Our analysis also showed that first name personalization is used much more frequently than last or full names, so we decided to see how the impact varied by industry. We focused on industries where the impact was significant, and found that there are several industries where use of the first name has a large positive impact. The most surprising finding, however, is that first name personalization has a negative impact on open rates for the legal industry.

“Free” isn’t guaranteed to help 

Does including “free” in your subject line entice potential readers to open your message? It does, but not always. The next few charts show that although the impact of using “free” in a subject was found to be positive and statistically significant for certain industries, it was smaller when looking at all industries on a higher level. Interestingly enough, use of the word “freebie” was found to result in a much larger increase in open rates.

 

The high level results suggest that using "free" doesn’t have a large impact, but we investigated at the industry level to see if that result is consistent across the board. What we found is that people in the medical, retail, and travel industries should avoid using the word "free," but restaurant and entertainment industries can certainly benefit from it.

People respond to a sense of urgency or importance

Needless to say, all of your emails are important. But there may be times when you feel like you need to use a couple attention-grabbing words to let your readers know that your most recent message requires immediate attention. It works. Words like “urgent” and “important” resulted in open rates that were much higher than normal.

Announcements, invitations, and cancellations

Recipients are much more intrigued by announcements and event invitations than cancellations and reminders. It would appear that repeated reminders and cancellations don’t pique their interest quite as much.

It might make sense that people don’t open emails about cancelled events. Sometimes the title says it all. Still, we wanted to know if this impact was consistent across industries. What we found is that using "cancelled" in subject lines is negative whenever the impact is significant—except for in the restaurant industry, where recipients seem more interested in reading on.

Requests for donations are largely ignored

Throughout the analysis, words related to charitable actions and donations had a negative impact on open rates. Although all these words negatively impact open rates, choosing the right ones can mitigate the detrimental effect of donation requests. Of all the related words we studied, "donation" had the most negative impact. "Helping" had the best impact, though it can obviously be used in more contexts.

Frequently used word pairs often have significant impacts on open rates

During our analysis, we identified a number of word pairs that were frequently used. Sometimes two words can provide context that a single word can’t convey. We thought it would be interesting to see how some of these word pairs perform.

First and foremost, people love to be thanked. It’s also apparent that campaigns about current events, like natural disasters and political issues, have higher open rates than normal. And finally, it seems as though recipients don’t like to be asked to sign up for anything—and they really don’t like being told they’re missing their last chance to get something they’ve already been emailed about.

Capitalization can help slightly

While performing the analyses above, we looked at the impact of capitalization on open rates. The results were quite surprising. First of all, the use of an entirely capitalized subject line resulted in slightly higher open rates than usual for a given user/list.

We then cast a wider net to see what the impact of having at least one fully capitalized word was, and the effect was slightly negative. Both of these impacts, though small, are statistically significant. It’s worth noting that our methodology compares campaign performance to other campaigns from the same user to the same list. That means completely capitalized subjects result in higher open rates for senders that also use mixed-case subject lines.

What does it all mean?

Many of these results can be applied to campaigns in a straightforward way. For example, we can say that these things are likely to increase open rates:

• personalizing subjects
• marking appropriate emails as urgent
• thanking your recipients

We can also say that choosing words wisely when soliciting donations or reminding your recipients of upcoming events can minimize the number of unopened emails.

The more interesting takeaway, though, is that a single word’s presence can dramatically alter the likelihood that your readers will open your emails. The content of your message is really what determines which words you use, but with so few words in a subject line, each one matters quite a lot.

MailChimp’s tracking options enable you to take a similar look at your own campaign history. Maybe you’ll notice a recent email that asked for "donations" rather than "help," or a crucial message that was labeled as "urgent" rather than "important." Basic campaign stats can help you figure out what works best with your own recipients, and A/B split testing can automatically conduct experiments to see which words work for your subscribers, following through with the best choice.

Above all, just remember that one word can make a big difference.