More than half a year has passed since Gmail announced the new tabbed inbox. My initial research hinted that open rates took a hit, but at that point I was just looking at a few weeks’ worth of opens. Now that things have settled down, we can look at months’ worth of all kinds of engagement data. To be specific, I pulled 29 billion emails, 4.9 billion opens, 4.2 billion clicks, and 43.5 million unsubscribes to find out how Gmail’s tabs are affecting open rates. (The Ops team wasn’t exactly happy with me for a few weeks there. Apparently, they’re very protective of our backup databases—and the backups of the backups.)

After looking over all the data, my conclusion is that Gmail’s tabbed inbox is working as intended. Open rates are definitely down, but there are reasons to believe Gmail users might be happy with the changes.

A matter of timing

One of the most telling stats about Gmail’s tabbed inbox is the time it takes users to open an email. I looked at every open for every campaign and measured the percentage of opens that come in minute by minute after the campaign is sent. The density of opens by minute tells us how quickly or slowly subscribers are getting to the emails in their inbox.

At first, I plotted every week since the beginning of May, and three distinct groups immediately stood out. Those groups lined up perfectly with the weeks before, during, and after the rollout of the tabbed inbox.

As a control measure, I applied those same time periods to some other email service providers (hereafter referred to as ISPs to avoid confusion). Gmail’s numbers are clearly a little wonky.

What you see is that during the rollout period, Gmail subscribers were actually getting to their emails faster than they normally would. That makes sense—there was a lot of buzz about the new inbox at the time, and everyone was excited to try it out.

After everyone got tired of clicking on tabs for the sheer fun of it, there was a marked shift in opens. Fewer opens come in around the 18 minute mark, while slightly more subscribers are opening their emails 40-55 minutes after the campaign was sent. Meanwhile, the other ISPs experienced little to no change. This tells me that Gmail subscribers are definitely using tabs to ignore certain emails.

Opens

Using the same time periods as above for "before" and "after," you can see the Gmail opens are down. My initial research encompassed a 6-week period around the release of tabs, but this data represents a 25-week period from May to October.

Opens from Gmail subscribers have fallen off nearly twice as much as any other ISP. If you’re surprised that other ISPs are also seeing lower opens, consider that MailChimp starts experiencing a lot of holiday-related volume in October. You might be familiar with a seasonal trend we call holiday email fatigue.

Clicks

To study clicks, I divided the number of clicks by the number of opens. That way, I could look at clicks in isolation from any changes to open rates.

I see another example of holiday fatigue in the data. It looks like Gmail click rates are down compared to Hotmail and Yahoo, but it’s not as bad as AOL or all the other ISPs out there. Maybe this has something to do with your ISP personality?

All in all, I’d call this one neutral for Gmail tabs. Clicks are down for everyone this time of year, and Gmail subscribers don’t look particularly affected.

Unsubscribes

This is where the data really started to paint a picture. Again, I looked at unsubscribes as a percentage of opens, and Gmail stands out as having no shift from May to October.

As a function of normal holiday fatigue, subscribers at other ISPs are starting to disengage with the newsletters they signed up for. They’re opening less, they’re clicking less, and they’re starting to unsubscribe more…except for Gmail. Gmail’s opens are down, the click rate is neutral, but unsubscribes haven’t gone up.

What does this mean? People unsubscribe around the holidays, because they get frustrated at the sheer volume of email hitting their inboxes. For Gmail subscribers, that frustration isn’t showing up in the data. If the data tells us a story, it’s that Gmail tabs are working as intended and helping people manage their inboxes.

Mileage may vary

Ultimately, the effect Gmail’s new inbox has on your email marketing depends on the relationship you have with your subscribers. I can’t create charts for each and every user, but I can narrow down the industries that are most affected.

 

The education and training industry really stood out to me. When kids go back to school in August, we usually see a huge bump in engagement to education and training campaigns. You can see that reflected from every ISP except Gmail. If you’re in any of these four industries, you might need to make an extra effort to reduce irrelevance.

At the end of the day, it seems people are using tabs to ignore certain emails, and it’s actually making them less frustrated with their inbox. Gmail would consider that a win. Is it possible they’re missing emails that would otherwise be important to them? Sure—but maybe they’re just getting to those emails a little later than they used to.

If you’re just catching up and want some context, read my July 23 post about Gmail tabs and open rates.