Please note that as of October 24, 2014, the Nokia Developer Wiki will no longer be accepting user contributions, including new entries, edits and comments, as we begin transitioning to our new home, in the Windows Phone Development Wiki. We plan to move over the majority of the existing entries. Thanks for all your past and future contributions.
Usability testing myths and facts
Including usability and usability testing in the development process is often considered expensive by the account managers. As they believe that instead of spending time and money on usability designing, the team should concentrate on the features and other aspects of the product. However over the years research and emperical data has proved beyound doubt that usability testing and designing for usability is indeed benificial to the company from a long term cost perspective.
Identify the tasks, which should derive from the use-cases prepared during the design process. The definition of task completion should be clear. Next plan a test schedule and choose a location. Select the representative users. Materials required for the testing process should be kept handy.
How many users to test with
Usability testing doesn’t have to be expensive, though. With only 3 to 5 users, you can find the majority of problems in your application, as has been advocated by usability guru Jacob Nielsen in his much respected and yet equally criticised article Why You Only Need to Test With 5 Users. Jacob asserts that 3 rounds of usability testing with 5 users is going to diagnose more then 85% of the usability problems. However there are other critics who point that 5 might be too small a size to look at for usability testing with the aim of covering more number of problems. They believe that depending upon the complexity of the solution the size of the usability testing group should vary. Laurie Faulkner for instance claims that 15 is a good number for the group size to look at, discussed in great details here in the article
Cost impact of usability
Another important to consider while deciding the size of the usability testing group is the cost aspect of it. Research has shown that adding more users to the group adds to the cost of the usability testing exponentially. So it is imperative you make sure that the group is not too big or too small for carrying the tests. The complexity and criticality of the product can also help decide the size of the usability testing group. For instance if the product in question is less tolerant to even minor usability testing you would need a bigger group as compared to one which is more feature focussed than usability oriented.
It is very important to do a thorough pilot test to make sure your task descriptions are understandable to others who are going to help you in the usability testing. You should lay down the specific areas if any that you are interested in being tested and a set of guidelines which could be used as a ready reckoners by the end users. Make sure that you include test users from every user group to your actual tests. That way you will get a wider perspective for your results as a similar group would give you a narrow if not one-sided feedback on the product. The test criteria and time taken to complete the tests may/should vary between different user groups. For example, novice users may be given a longer time to complete some tasks than expert users.
What do you gain from usability tests, then? You can see if there are features that the users can’t find. You will find out if they can understand the terminology used, or if it takes too much time to find the right keys or features? Most importantly you will find out if the users can actually complete the tasks they are asked to do and if they are satisfied with the experience of using the application. Usability testing improves your application’s quality!
--- Edited by Mayank on 22/06/2009 ---