Aug 27, 2009
Gmail users more engaged than Yahoo, Hotmail, AOL?
We recently analyzed the stats for over 184 million emails sent from MailChimp and put together a report analyzing the engagement of recipients by email provider (yahoo, gmail, hotmail, aol, and comcast). We wanted to know if certain subscribers (such as hotmail users) could be expected to respond any differently than, saaaay, gmail users.
Here’s a summary of what we found:
| Domain | Open Rate | Click Rate | Soft Bounce Rate | Hard Bounce Rate | Abuse Complaint Rate | Unsub Rate | Sent |
| Yahoo.com | 24.54% | 4.17% | 0.08% | 1.09% | 0.19% | 0.35% | 54,791,998 |
| aol.com | 20.09% | 4.25% | 1.48% | 2.92% | 0.32% | 0.51% | 28,750,743 |
| gmail.com | 30.94% | 7.41% | 0.13% | 0.28% | N/A | 0.50% | 28,997,678 |
| hotmail.com | 23.79% | 4.49% | 0.31% | 0.80% | 0.24% | 0.43% | 63,465,012 |
Some things we found interesting:
Gmail subscribers seem to be a bit more engaged than the other subscribers. Could be some demographic kinda thing.
Also interesting to see that the number of emails sent to gmail has surpassed AOL. Interesting to me, since AOL has had a head start over gmail, yet Google has caught up in just a few years. Keep in mind this is just emails sent. I have no idea what the "number of subscribers" are at those domains (nor does TechCrunch, but this article is still somewhat interesting).
Oh, and the reason there are no abuse complaint numbers for gmail is not because those wonderful recipients never complain. It’s because we currently have no way of measuring complaints, as gmail has no feedback loop.
In terms of emails sent, yahoo and hotmail are both still significantly higher than gmail and aol.
For all you stats-freaks w/calculators, I’m leaving out a row of data from comcast.net (which represented 8,673,998 emails sent). Frankly, it was so out of line with the others, we’re going to look into that one in more detail. The data set we analyzed was "all emails ever sent from our servers" but excluding campaigns sent to lists smaller than 2,500 recipients.
And if you’re interested in these sorta stats, be sure to check out Chimp Charts, where we post free email marketing data like this quite a lot.
Does this make you wonder what the composition of your own subscriber lists look like? MailChimp users get a free domain performance report for all campaigns.
Want to see if certain subscribers on your list respond better than others? Try some simple list segmentation.
Robert Rawlins
Really cool stats there, interesting that it’s by such a massive margin, the other 3 follow a fairly uniform trend but that is indeed a real stand out figure from gmail.
I think I’d probably support your guess that this is a demographic thing, I mean, who would be caught dead using AOL, honestly? haha unfortunately we know there are at least 28,750,743 of them ;-)
The only other motive I can think for the higher open rate in gmail is perhaps an anomaly in the junk mail filters of the other platforms pruning them out so the number of users with the mail visible in their inbox is smaller? I don’t know, or has that been accounted for?
We have almost identical challenges here with our product, we can quickly analyse and compile delivery data for our content but trying to understand the differences between cell phone manufacturers and locations is always a little difficult, we always try to factor in as much data as possible, weather conditions, time of day, demographic, there’s only so far you can go though before it makes your head spin.
Interesting stuff guys, as always.
Rob
08.27.2009
Nicole
Do you think it has anything to do with the gmail service being newer? I, for example, set my gmail account up (which came many years after my Yahoo one, of course) to be my “clean” account. I use that address only for things I’m particularly interested in. I send everything else, and those I suspect might oversend, to my Yahoo account. I know others who also do this, so I suspect I’m not alone here. Thus, since I send only things I’m particularly interested in to Gmail, I open it, and thus appear more engaged.
Conjecture, sure, but a possible explanation.
08.27.2009
Ben MailChimp
Hi Nicole, yes, I suspect that’s a big factor here.
08.31.2009
George Bilbrey
Can you define “Open Rate” and “Click Rate” – how do you calculate?
08.28.2009
Ben MailChimp
Hi George, opens are counted w/out the bounces, if that’s what you’re wondering. We don’t include those when we run the calculation. Also, a click is counted as an “implicit open” so that we’ll register opens even for plain-text messages.
08.31.2009
Edwin
Not sure I understand how you define an open. In my experience, we embed an image in the mail we sent out and track the # of times the image is opened. That’s our definition of “open”. As such, I am not sure how you can measure “open” for plain text email. Also, all these domains you included have image displayed off by default, so using my methodology, the open rate tends to be artificially low while the CTR tends to be artificially high. So, sometime, we look at the click to send ratio to put them on equal ground for comparison.
09.10.2009
Ben MailChimp
Edwin,
Re: the measure of open rate for plain-text emails – clicks are measured as inherent opens. And yes, those domains do indeed have images off by default, but it’s all of them, not some of them, so it’s a fairly level playing field in that regard. Looking at click rates are very important indeed to keep proper perspective. You might also check out http://blog.mailchimp.com/charts for more stats.
09.10.2009
Mike
Hi Ben,
I was wondering if you could respond to a couple methodology questions I had concerning this data. When did the e-mail campaigns take place and were they US based or global?
Thanks!
09.01.2009
Gurumustuk Singh
I know many users moved to gmail and like me have old hotmail/yahoo accounts that I no longer use. They are just junk mail boxes now piling up and never read.
09.04.2009
Jeff Lowe
I have a theory on this…
In my experience Gmail does a far better job at detecting and correctly filtering spam than the other mail services (I have used nearly all of them).
I use Gmail on a daily basis and would say that only 1 out of every 40-50 emails that ends up in my inbox is spam. With other email services it was at least a 1-1 ratio.
So my guess is that users of the other services are so used to seeing spam in their inbox, that they skim their messages (mentally categorizing any email not from a known sender as spam).
Just a thought.
09.07.2009
Mick Watson
This is the initial thought that came to my mind. I have a very good experience of spam protection with gmail and for that reason I tend to find that my guard is noticeably lower.
I find that I’m much more likely to give a chance to that sender I don’t remember or to that slightly spammy subject line, within reason. I’m not entirely sure whether this is because I feel that having passed Gmail’s spam filter it’s worth my time, or just a product of the fact that having less spam means I have more time to risk.
09.15.2009
Kamran Razvan
I find your results quite interesting and in-line with our findings. I found your post trying to find other data on the web regarding user traits with different email clients.
You can review the analysis from 2 years of data at Click & Pledge through the following link:
http://techscoop.clickandpledge.com/2009/09/click-pledge-part-1.html
If you are interested we can do some joint analysis of user behavior together. I think such analysis can benefit our mutual clients.
Regards,
Kamran Razvan
Click & Pledge
09.23.2009
Ben MailChimp
Kamran, that’s really fascinating! We’ve pondered doing a similar analysis, but through our ecommerce360 plugin (email revenue by domain). We’ll be passing your link around for sure. Thanks!
09.24.2009
Floran
Hi Ben,
When people open a message in their mailbox in general they double click on the Subject or the Sender of the message, right? When you single click you would normally select the message. In this way you are able to multi select (with shift and ctrl) messages to delete them.
Can the open rate be higher in Gmail due to the fact that in Gmail the subject of a message is a “link”, so when you single click it, it opens that message…
I have Gmail and find it sometimes easier to just open and go back to make a message appear as “read” then to select it at the box and delete it or mark it as “read”….
Conclusion I draw is that it is in the nature of Gmail that you open a message…
What do you think?
Regards,
Floran
11.12.2009
Chris Davies
Hi,
Going back to the question of a definition of an “open” email, is “opening” an email in the outlook preview pane considered an “open”? I rarely ever double click on an email in outlook so that it opens in a seperate window. I understand that clicking on a link via the preview pane, is an implicit open as well.
Would appreciate clarification on this and I apologise if it’s been clarified in another section.
Kind Regards,
01.14.2010
Ben MailChimp
Chris – Whether someone previews or full-views, an open is only tracked if they download the images in that email. Open tracking is down with little transparent tracking .GIFs that are downloaded from the server. Alternatively, if someone chooses not to download images (which is frequent), but they click a link, we’ll track that as an open.
01.14.2010
kensley
Ben, when you say “but they click a link, we’ll track that as an open”, you mean when they click the link to open the e-mail, as apose to going down your list of e-mail links listed correct?
09.18.2010
Ben MailChimp
Nope, they have to click a link inside the email content. If they do, we consider that an inherent open, and will track that.
09.18.2010