How NOT To Conduct Usability Tests

Usability testing is a way to evaluate how your target users are using an app, website, or any other piece of software, during development, long time prior to launching it into the greater world. A good test designer has to be aware of the pitfalls that could interfere with the outcome of the test. Keeping in mind these problems is certainly the best way to stay clear of them.

In this article, I present to you five main concerns that can derail a usability test and which you should try to avoid when conducting your own usability tests.

5 Main Concerns that can Derail Usability Tests:

1. Lack of proper timing

Many developers tend to conduct usability tests at an advanced stage of production. Keeping your hand constantly on the pulse of your project is the way to maximise the effectiveness of usability tests.

Avoid conducting a battery of usability tests all at once; rather, you should time your tests at appropriate moments in the development process.

When you carry out usability tests too late in the project, you run the risk of having to fix certain problems by starting the process from scratch. Catching usability issues early on is the best way to use tests; timing the tests beforehand at significant points in the development process enables you to obtain reveal precious feedback which you’ll be able to act upon sooner. One good solution is to conduct usability tests in each and every sprint.

Timing is also to be considered as the testers who normally work in the business might have other priorities. Even though usability testing is important to you and the project, the users are understandably more likely to prioritize their daily work higher. According to our experience, tests should take not more than one hour to perform. One hour is often possible to squeeze into people’s stressed daily work without taking their day over.

2. Unrepresentative user sample

Sometimes it’s tough to find a good user sample, and small companies or start-ups may have to rely on the same group of people over and over. It is easiest to test on team members or your sales department, but these people are biased and most likely not very representative of your target group of users. Usability tests are most effective when they’re carried out with a sample that represents accurately your target user population.

Try to vary the mix of users you obtain feedback from. Smaller companies can ask friends to refer other friends whilst bigger companies could offer incentives to attract new users to participate in testing. The people you select for a usability test are as important as the test itself.

If you want to make the best out of a usability test, I suggest you give yourself ample time to find a user sample that matches your targeted demographic. If you use personas, use them as a base for selecting testers, Don’t conduct usability tests in a rush. This ties in well with the point mentioned earlier; timing your usability tests well allows you to plan your recruitment process adequately and find good test subjects.

3. Poor test setup

So, you’ve got the timing right and you also have a motivated group of users ready to test out your work. But how well have you planned out the actual usability test? Conducting a test without clearly-defined end-goals can lead to important data being overlooked or unusable feedback being collected.

Image courtesy of userlytics.com

You cannot conduct a usability test before deciding what is its purpose in relation to the main project. Do you want to explore in-depth the users’ experience of interacting with your product? Are you looking to assess how your app or website responds to the users’ demands, or to validate how well your app or website fulfils its goals? Each question requires a different type of test, and a corresponding class of data to be harvested from the test.

However, correct test setup doesn’t depend only on the test design you adopt but also the type of environment you are conducting it in. Don’t carry out usability tests in places that are noisy, cramped or will cause distress to the participants. Sometimes this is difficult for smaller companies to work around, however testers have to be considerate towards their participants’ needs and try to find an adequate location where to conduct a usability test.

4. Unreliable testing methods

Almost as bad as an unfit testing venue is conducting a usability test without the proper tools to capture the data you’re looking for. Choosing a good testing software is an important step in ensuring that your usability test results are reliable and accurate.

The testing method you’ll use needs to be simple and easy for your participants to interact with. Give them the time they need to familiarise themselves with the test and try not to interrupt them too often while the test is in progress. Use open questions that can’t be answered by a simple yes/no.

Of course, as a test leader you need to be thoroughly familiar yourself with the testing software and the recording method you’ll adopt for the usability test. Don’t go into a test unprepared. If you have enough time, conducting a smaller pilot test can help you pinpoint any issues that should be dealt with and resolved before the actual study takes place.

5. Bad attitude

Making people feel bad is a good way to spoil usability tests. It is important that you explain the purpose of the test: to learn how users interact with the system, not to test the users’ ability to understand. Write down the instructions so the user can relax and read through before starting the test.

This one might not be high up in a developer’s lists of priorities, but your attitude while conducting usability tests can influence greatly the quality of the feedback you obtain. After all, the users you recruit to carry out the test are giving up some of their time to help you!

I think approaching a usability test with a good attitude is one of the most important factors in a test leader that contribute to the success of a test. And a truly positive attitude (as opposed to fake cheerfulness) is possible only when all the other points mentioned here are fulfilled.

You had time to prepare for the test, you got a representative sample, you’re aware of the exact outcome you’re looking for in the test, and you implemented the best tools to capture that data.

There you go; you’ve got four good reasons to smile.

Conclusion

In this article I covered the five things to avoid when conducting your usability tests. Make sure to plan your test well beforehand – pencil it in your production schedule – and avoid timing it badly. Follow that up with a comprehensive user recruitment schedule; don’t just grab your buddies every time you need somebody to participate in a usability test.

Choose your tests wisely. Formulate your problem-question carefully and then find the best test to answer it for you – don’t fall into the trap of carrying out the same test every time. When the users step into the testing room, ensure that they feel comfortable and that your recording instrument is up to scratch. And finally, remember that your attitude can do a lot to influence the end-result of your usability tests.