Usability testing can shines a spotlight on parts of your product which might confuse or mislead users. Of all the types of feedback you can get from customers, usability feedback can be the most tedious to obtain, yet also the most concrete. It helps clarify whether real people will actually be able to use your real product…and whether they will want to keep on using it.
Sit down with your favorite user experience designer, buy them a cup of coffee (maybe even a muffin), and ask them about usability testing. Then let the debate begin about when, how, and how often to perform usability tests and whether conducting a usability test is a professional skill or something amateurs should be doing.
When should you gather feedback via a usability test? “If you want to learn how quickly and effectively a participant can complete tasks with your product, then you should run Usability Testing. Watch your participants use your product to see where they are successful and where they struggle in order to improve the user experience.”
In order to run a usability test, though, you need to have a working product. Paper prototypes, non-functional html mock-ups, buggy beta versions…they will get you some feedback, but will be skewed compared to the findings from running usability tests with working products.
Usability tests will classically show you how real users interact with your product. What areas do they intuitively understand? Where do they stumble? Where do they do things you did not expect them to do, or take actions you had not anticipated?There are two very valuable parts of this feedback.
Collecting the Feedback: Moderate-Difficult. Usability Tests are often intimate, one-on-one conversations with users, where the user is given some context and then asked to complete a task (or tasks). You need to spend the time to plan out a usability test, and conducting them takes time. Even if you’re conducting remote or automated usability tests, set up takes time.
Analyzing the Feedback: Easy. Assuming you have taken the time to plan out the test, tasks, approach, hypothesis and metrics, assessing the success or failure of a usability test is relatively easy.
Reach: Deep and Narrow. Usability tests are generally focused on optimizing specific tasks.
Scalability: Difficult. These tests are typically one-on-one conversations. Not so scalable, even if they are scripted.
Cost: Varies. You can run usability tests on the cheap by simply giving a user access to your software and remotely observing what’s on their screen. Alternatively, you can hire a firm specializing in this sort of thing (bigger bucks) or use an automated tool.
At what point(s) in the PDLC will this type of feedback be most useful?The Product Development Lifecycle:
Tapping into Usability is helpful only at specific points in the Product Development Lifecycle.
To get better results, don’t just rely on usability testing or analytics, let them inform each other. As Ryan van der Merwe of Jive Software writes, “in order to know where to start to improve the user experience of your product, you’re going to need more than a single data source. No single method can tell you the whole story you need to make good decisions. Usability testing will show you where there are interaction problems, but it won’t show you the magnitude of those problems. Web analytics will show you lots of numbers, but it can’t tell you how to fix issues you encounter.” He goes on to make the case for the two approaches: 1) conducting usability studies first, then using those results to home in on areas worthy of further analytics or 2) start with your existing analytics to develop a hypothesis and use usability testing to validate it.
There is some great advice in the post User Testing isn’t (Always) Rocket Science, and the most important theme is that until you see how people use your products, you cannot always predict what will work or why they might fail. “‘Words and phrases matter to the users. With content, you think you’ve got it nailed down, then you see users get stuck on something really simple. Without hearing [the feedback] on the way, we wouldn’t know why the site was not working later…It’s hard to drill down into why people are doing or not doing something. It’s hard to do with just analytics. When you hear and see people react, you can get into their heads and make decisions based on this.’ says Margret Schmidt, VP of User Experience & Design at TiVo.”
Dana Chisnell, co-author of Handbook of Usability Testing, agrees, “I contend that 80% of the value of testing comes from the magic of observing and listening as people use a design. The things you see and the things you hear are often surprising, illuminating, and unpredictable. This unpredictability is tough to capture in any other way.”
Anthropologists and ethnographers make a point of observing their subjects in their natural habitats, since they can learn so much more when they have the surrounding context. The same can be said of usability testing: you can conduct a usability test in a sterile usability lab, yet imagine how much richer the results will be when they are in the context of how and where the subject will actually be using the app. Test a shopping comparison app in the aisles of a store (poor lighting, dodgy connectivity). A sports gamecast app in a bar or stadium (extremely loud and incredibly close). A CAD program used on a construction site (large plans viewable by many hard-hatted stakeholders on an itty-bitty mobile form factor). What works well in a contained usability lab may fail in the real world. And unless you’re creating products that will be used in an actual lab, they will not actually be used in a lab setting.
That said, there are other ways to gather real usability feedback from real users. Services such as UserBrain and Usability Hub might just be the next best thing, since the feedback comes directly to you.
During usability tests, ask test subjects to narrate what they are thinking and doing as they do it. Rather than trying to divine their intentions by asking a lot of questions, you can allow them to provide the talk-track. Prepare to be surprised and to follow up once the task is completed with additional clarifying questions.
If Big Data is about gleaning macro insights from many micro data points, Usability Data is about discovering micro insights from a few macro interactions. And in the world of usability testing, you only need a few data points to draw a substantive conclusion.
“As you add more and more users, you learn less and less because you will keep seeing the same things again and again. There is no real need to keep observing the same thing multiple times, and you will be very motivated to go back to the drawing board and redesign the site to eliminate the usability problems. After the fifth user, you are wasting your time by observing the same findings repeatedly but not learning much new,” says Jakob Nielsen in an article on the Nielsen Norman Group blog.
Usability tests provide feedback on how real customers use particular parts of your product to complete specific tasks. By watching them perform the tasks, you can learn whether your product is well suited to the task, and how it might be improved. Usability tests cannot tell you if your business model or price point are correct, but they can validate if you’ve designed your product to be easily useable. They will help you understand why users are (or are not) continuing to use it and how you might be able to make it better.