I talked about doing a little mobile testing a few days ago, but that was just an exercise in learning how infuriating a non-mobile-optimized website can be. That wasn’t what I’d consider true "mobile app testing." The phones were mounted on a fancy camera rig, and the environment was totally artificial (for better filming).
In reality, we use our devices at home, at the grocery store, while walking, while driving, and while brushing our teeth. We spend more intimate time with our phones than with our spouses. Seriously, we keep them right there in our pockets, next to "that place where the sun don’t shine." They’re very personal, and therefore testing them has to be very contextual. Which is really hard (see: Designing the Mobile User Experience).
To test a mobile app, you have to get intimate with it for a while. A few of us have been living with Gmonkeys (our upcoming iPhone app) for the last few weeks, and I thought I’d post some observations.
But first, a little back story.
Over the summer, we decided to get serious about mobile. We had already dabbled with a mobile version of MailChimp back in 2008, and later hired Pinch/Zoom to build our first native iPhone app. They did an outstanding job, especially considering they’re not an email marketing company. And this early experimentation taught us a lot about mobile, allowing us to build some innovative and powerful mobile features into MailChimp.
But we’re noticing big, disruptive shifts in the way people use the web. Social is powerful, but when combined with mobile, it’s game changing. It really changes your expectations about the social web. So we decided to start investing in this mobile stuff.
Developing a taste for dog food
When nerds talk about "using your own product" they call it "eating your own dog food." We figured if we’re going to get serious about mobile, so that our customers can get serious about mobile, we better start eating. First, we noticed this really smart guy who used the MailChimp API to build a Blackberry app called MiniChimp. We suckered him into joining us full time (welcome to the team, Eric).
Then we implemented a program where every employee at MailChimp gets a free smartphone. We cover the device, and we cover the monthly fees. Interestingly, most people got iPhones, a bunch of people got some Droids, and a couple people got Blackberries. It’s been interesting to see how people are using them, too. Check out Blaine, from our customer support team, tethering his Droid to his iPhone in the Grand Canyon.
Then we started our Mobile Lab (you may have seen the announcement in our Summer Newsletter). Amro and Drew are entrepreneurs who’ve been building quick and useful little mobile apps for their clients, and also happened to use MailChimp for a few projects. We hired them to help us with Chimpadeedoo (for the iPad), then with ChimpKit (a MailChimp API wrapper for mobile developers). We liked their work ethic so much, we tricked them into joining us full time too (welcome to the team, guys).
Living with monkeys
Which bring us to our Gmonkeys app. If you haven’t read about it yet, here are some details. Basically, it’s an app that sends you push alerts (in near real time) about important customer activity on your list. You probably have a handful of customers that you watch very closely. Maybe they’re your biggest spenders, or biggest "influencers." In my case, maybe they’re the ones who give you the most feedback about my company. Gmonkeys (short for "golden monkeys"), is a tool that tells you when they interact with your campaigns.
A handful of us at MailChimp have been living with Gmonkeys for the last few weeks. I’m realizing in the process that testing a mobile app is not quite like testing a web app. Most importantly, I don’t have easy access to bug tracking or reporting software when I’m on the go (heck, even when I’m at home on the couch). So if I experience something quirky, and I’m away from my desk, I have to jot it down somewhere or make a voice recording or something, before I forget.
And instead of simply reporting, "I got a weird result while using Firefox on a Mac when I clicked this link" and then attaching a screenshot, I have to describe the scenery (remember, "context is king" with mobile).
For example, a week ago a big storm swept through Atlanta. So around 4:30am I was awakened by lightning, deafening thunder, and BLOOD CURDLING SCREAMS echoing down my hallway. Turns out it was the Gmonkeys app.
See, it’s supposed to give a cute little monkey-ish "eep eep" whenever an important client clicks a link in my emails. I wanted it so that if I were in a meeting, it would be loud enough for co-workers to hear it. Yes, I actually tested for that, and it works. It’s an excellent conversation starter. "Oh that adorable monkey sound coming from my pocket? That’s a Golden Monkey. See, this VIP just opened my email newsletter. Let me show you how awesome this is…"
Turns out that cute little eep-eep is not so cute if you check your email one last time while brushing your teeth, and you leave the iPhone on the bathroom countertop. The flat back of the new iPhone4 sits perfectly flush with the countertop surface, turning it into a gigantic speaker. So when the push alert arrived because someone opened my RSS-to-email campaign (which is scheduled for delivery every morning at 4:00am), the vibration of the phone PLUS the monkey screeching scared the bejeezus out of me, my wife, and my 2-year-old son.
Interestingly, I wasn’t mad about it. Instead, I ran to the phone to see who the heck it was (it’s that addictive), and why he might be opening my email so early in the morning.
I opened my MailChimp account, searched for him on my list, and our geodata shows that my customer actually lives in Spain (and 4am Eastern is about 10am in Madrid).
And that’s when I got the idea: when we send a push alert about someone, let’s just display their geolocation and their local time right there in the app.
That little detail, and a few others we’ve been adding to Gmonkeys, probably seems obvious. But it’s the kind of stuff you only think of when you truly live with the app and eat your own dog food.
Help Us Beta Test?
If getting awakened in the middle of the night by screeching monkeys sounds like your kinda fun, why not sign up to be a beta tester? We’ll be selecting a handful of customers shortly.