A good friend paid me the ultimate compliment last week. Responding to my new website, they described it as “nice and simple”. Resisting an urge to go the full Jobs on him, and instead reaching for a higher power – I replied simplicity is the ultimate sophistication. If it’s good enough for Leo it’s good enough for me…
Anyway, I promised you we’d keep looking at learnings from my new website. If last week was about the perils of being in the middle, this week I want to focus on testing, and the insanely simple ways you can create a much better product – often with less testing than you might think necessary.
Testing is important in several elements of your product, perhaps most obviously on your assumption and then on your design. Basic testing can get you a long way to “good”, even if many creators forget or ignore this. The seriously good news for the smaller businesses amongst you is testing does NOT have to be a laborious and expensive process. Just how far from costly may in fact surprise you.
Research from Jakob Nielsen and Thomas Landauer has unearthed an unexpected (for me) result. Elaborate usability tests are often a waste of resources. The best results come from running multiple small tests, across a diminishingly small test group. Or as the brains put it, N (1-(1- L)n).
Nope, me neither. The bit I did understand was this – you can uncover up to 80% of your product’s problems simply by testing with 5 people.
Here’s how Jakob describes the process:
The most striking truth of the curve is that zero users give zero insights.
This then moves up the scale
- As soon as you collect data from a single test user, your insights escalate and you’ve already learned almost a third of all there is to know about the usability of the design
- When you test the second user, you will discover that this person does some of the same things as the first user, so there is some overlap in what you learn. People are definitely different, so there will also be something new.
- The third user will do many things that you already observed with the first user or with the second user and even some things that you have already seen twice. Plus, of course, the third user will generate a small amount of new data.
- As you add more and more users, you learn less and less because you will keep seeing the same things again and again
Over the years I’ve been guilty of great designs that have come unstuck by being dropped into the hands of customers untested, partly due to my rush to getting it into market and partly (naively) trying to save time and cost. Part of my learning has been the value of testing as you develop – doing this can help you proactively fix the problems as they are unearthed, which saves both time and cost.
The primary reason problems appear as soon as use your product is we all have an inherent biassed worldview. It’s extremely difficult for your product to escape that unless you expose it to real customers as early as possible. There are many reasons to convince yourself you don’t need to – you live and breath your business and your customers don’t, you have a passion for your new products and services and your customers don’t… but none of these are real and viable.
The reality is customers don’t care about your business anything like you do – they care about you solving their problems. What may be intuitive to you may be completely foreign to them. Watching a customer or user use your product for the first time (through a two-way mirror or on video) can be frightening – they just don’t get it, despite the fact for you “it” is right there in front of them… why can’t they see “it”… and finally, come on, how hard can “it” be???
The truth is, bloody hard, and all because you designed it with your biassed worldview. The only way to get into your customers’ mindset is through testing.
Needless to say – in an international liquid workforce market various businesses can do your testing for you. It can be as low as US$99 per user to have this done, UserTesting has a great sample video of their process here.
I didn’t hire usertesting.com for my site, though I certainly tested it. I followed Nielsen and Landauer, sent it to five people (all acquaintances or customers) I knew were honest enough to be honest with me, and let them loose. And they found a lot of things that either weren’t right, or didn’t read how I thought it would, or just plain didn’t flow. It was a simple process but added immensely to my final design and content.
Key Takeaways
- Zero users give zero insights
- It only takes five customers to give you 80% of the insights
