Feb 1, 2012

Keeping Our Eyes on Video

A few years ago MailChimp decided to take video seriously. Well, in the beginning, the videos themselves were never very serious, in fact, quite the opposite. But they have always served a very serious purpose, which is to help our customers learn how to use MailChimp, learn about new features and learn about our awesome customers.

But how do we know if these videos are doing their job? I get asked that a lot. Well, it’s all about the stats.


Graph of viewership, spiking the day we emailed the Wavelength video.

A big part of the using-video-on-your-website mix is the video hosting service that you use. We’re a long way from 2005 when there was either YouTube, or an .flv or .mov embedded in your webpage. Today’s video hosting landscape is more like a food court at the airport; Do I want some cheap fast food? Or some cheap fast food posing as cheap healthy food because its on flatbread? Or do I want to spring for that place that looks all dark and leathery with lots of TVs and the $18 hamburger? There are plenty of choices out there, but making the choice of who to go with is not always so cut and dry.

We’ve tried out more than our share of hosts these past few years. All have had their pros and cons, their sweet spots and their misgivings. But this year we’ve moved our content to a new home that looks finally like a place to set down some roots. And that place is Wistia.

Along with a long list of the necessary fundamentals, such as solid service and performance, good support, and an easy workflow, Wistia provides some fun and helpful performance metrics that help us know how our videos are performing, and where there may be room for improvement.

One of the minor miracles of the internet and the video hosting revolution is that you have access to an abundance of statistics that can show you, down to the second, what your viewing audience is reacting to, and how. If you aren’t convinced that this is truly miraculous just do a little reading about the complexity and cost of the Nielson rating system for TV shows (viewer diaries, "Home Units" and "people meters"!), then realize that with the internets we get at least the same amount of information as they do for a minute fraction of what that system cost in terms of time, money, and effort.

Recently we sent an email campaign to 1.2 million users that announced a new service called Wavelength. In that email, we linked to a video hosted on Wistia, which meant we could go in after the fact to see how it performed. With Wistia’s metrics, not only can we see the normal stuff like total loads, total views, and average engagement,

The basic video statistic summary.

but we also get to see these nifty little "heat maps" that show each viewers engagement in a neat new way. They also show where in the world the view is from, which is always pretty cool.

Some Wistia heat maps.

The heat maps display a hotter color within the timeline of your video as people scrub or rewind to rewatch any portion of the video. So in a loose sense, we can look for patterns that may tell us what particular points in the video people needed, or wanted to see again. We can also see gaps in the timeline if they skipped sections. Of course interpreting these graphs is a fuzzy science, but if there are similar spots across the viewing audience that see more heat on the map, we can look at that point in the video and consider what was either extra attention grabbing, or maybe extra confusing, our maybe something so incredibly cute that people just couldn’t help but to watch it over and over again.

In this case, there wasn’t a strong pattern of viewing a particular point over and over again, but there was a telltale pattern of fall-off around the 20 second mark. After watching the video again I can see that in the first 20 seconds or so we define what Wavelength does in general terms. After that we launch into more detail. So, many people watched the first twenty seconds and got the gist of it, then switched off.

Statistics showed that 82% of the video was watched on average, which is actually pretty darn good for a web video that is 1:30 in length. If that number was significantly lower, I would be more concerned about that dropoff pattern 20 seconds in. I would be inclined to go back and rework the script or the treatment to encourage viewers not to leave. How would I do it? Maybe have the narration hint at something coming up later in the video, or possibly add a surprising or entertaining moment which may rekindle interest and buy us a little more time with our viewer. As more people are used to seeing and watching video on the web, convincing them to click the play button is getting easier. Convincing them to stick with the video, however, is actually getting harder imho.

All these cool metrics can really help us do just that. It’s simple enough these days to go back into the video and retool it as needed, then export a new version and swap out the last one. This way we can continue to iterate, update, and improve our content as needed. And in my experience, it’s rare that something that is ultimately a subjective piece of creativity, such as a video, gets so much direct objective feedback from a broad audience. So I’ve really found these stats to be a huge help as I’ve continued to create and shape the videos we produce. They’ve also thickened my skin a good bit.

Another nice thing about Wistia is how we also get a lot of control over the look and features of the player.

Simple player customization tools.

Many hosts allow you only limited control over what color your frame and buttons are, and what controls you can include. It’s nice to finally have an easy way to make the player look as minimal as we want it to and to be able to color it to match the palette of the page it will live in. It would have been a big bummer to have a bunch of big, off-color controls covering up the eyes and our pretty new logo in the poster frame of the Wavelength video.

As it so happens, we first learned about Wistia because they contacted us a few years ago when developing their own API thingy between MailChimp and Wistia. It allows you to integrate your video with your email campaign, which is pretty cool . And MailChimp has a Wistia merge tag which allows you to integrate a Wistia video of your own into your email newsletter and take advantage of the awesome statistics. Read about it here.  Ben also just wrote a blogpost about how we allow you to customize and automate your video merge tags.  Pretty simple stuff to do, but as you can see below, it may make waves if you actually use it:

Clickmap of an email newsletter.

This is a MailChimp click map of the email Ben sent out announcing Wavelength. As you can see, click rates on the text links averaged around 4-8 %, but that video staring atcha there got a whopping 62.8% of all the clicks. Maybe it’s that hypnotic eye power getting people to watch (I have a proprietary "hypnotic eye power" filter; merge tag coming soon!), or maybe it’s people’s inclination these days to prefer watching a quick video instead of, or in addition to, reading about something. Whatever it is, it’s these insightful statistics that help prove this video thing is worth keeping our eyes on.