Please note that as of October 24, 2014, the Nokia Developer Wiki will no longer be accepting user contributions, including new entries, edits and comments, as we begin transitioning to our new home, in the Windows Phone Development Wiki. We plan to move over the majority of the existing entries. Thanks for all your past and future contributions.
Series 40 UI design usability testing webinar - companion article
This article is companion for the Debug your design for Series 40 Full Touch webinars held in December 2012. It covers both sessions: 11th and 12th December 2012.
- Series 40 UI design usability testing webinar - companion article
- Series 40 UI concept design webinar – companion article
- Series 40 UI game design tips webinar – companion article
- Series 40 UI design port from Android webinar - companion article
- Series 40 UX checklist webinar - companion article
- Series 40 UI design for monetization enablers webinar - companion article
You know it's important to test your app before you publish it, but if you’re testing only your code, you’re testing only part of your app. It’s important to find bugs in your design as well. Otherwise, the app may behave exactly the way your code intends, but users may still find it frustrating and give it low ratings in Nokia Store. Join UX expert Jan Krebber of Digia as he presents simple methods to find UX bugs and shows you how to correct them. These techniques can help you increase your app quality whether you work with a large organization or completely on your own. As usual for Nokia Developer UX webinars, the presentation will feature exercises that will receive follow-up treatment in this Nokia Developer Wiki article.
The webinar and the wiki article are a great starting point if you’re not a UI design expert. Check out Series 40 UI design library for more advanced information and resources.
This wiki page is the "companion" to the webinar. It includes:
- Webinar exercises and proposals for how to solve them (will be added after 18.12.).
- Checklist example from the webinar
- Time plan example from the webinar
- Open issues from Q/A
- Further links and references
- Slide deck link
- Recording link (will be added )
- Webinar announcement
Proposals and solutions to webinar problems
This section contains problems raised in the webinar exercises. In addition, it contains proposals and possible solutions to solve the problems.
Watch the videos below. What went wrong? How do you fix it?
The media player is loading...
- She does not find the intended way of creating a new item via the list item.
- She sees the text
- But she is reluctant pressing it
- She states that she would have expected an "add new" button in Action Button 1 and not as list item (0:50)
- She goes via the Options menu to create a new list, which is more work than intended
- As long as "create new list" is not becoming it's own button, people may miss its functionality
- Change the list item "Create new list" to Action button #1.
The media player is loading...
- More information is not anticipated
- She is wondering from where to get more information before she buys the item
- Not evident from the layout that there is more information after the next step
- Expects to buy the item after the 2nd screen, but she does not know how much will cost
- It says "buy" even the next step still shows some information
- The price information is missing
- Buying process broken at the end
- She thinks the process got broken since she did not see/know where her item went
- Wants to start the purchase process again
- Inform people that there is more information after this view before they actually buy
- Do not show "buy" here, but mention the price at this point
- Buy should be reserved for the last screen before the shop UI takes over
- Change the purchase flow so that the user sees the map download. This could be even an "artificial download indicator" just long enough for the user to see where it went, that it got downloaded.
When are 2-3 participants sufficient for usability testing? Why?
- If you test multiple times, e.g. in a 8 week mobile implementation phase with 4 sprints could have 5 tests (one before and 4 after each sprint) and 2,3,2,2,3 participants, that yields to 12 participants
- You will not find all usability problems in the first test round with 12 participants yo will spot the majority of your problems
- Frequent tests will allow you to re-test some of your redesigns or tweaks
- With more participants (4-6) you will not learn much more during one round
You are not sure what a participant is thinking. What do you say?
- What are you thinking? :-)
- What are you doing right now?
- Is this what you expect to happen?
A participant mentions that she is not giving you what you expect from her. What do you say?
- Even if you think this is not the right information you were looking for, it can be valuable information. You must collect it and analyze later, after you have a clear mind.
- It is important to keep people talking and not to scare them away but make them feel comfortable.
- Even if the current information is completely useless for you, you must keep the participant in a comfortable mood. Maybe she will will point out some important stuff later on.
- Value participants' input, value their help, value that they give you their time.
- You must have a positive attitude about all the information you are getting from a test. The information is there to improve your application and to help you in the end to improve the Return-of-Investment (no matter if this is real money or credits).
- You say:
- No worries, this is exactly what I am looking for.
- I haven't looked at it from that point of view before, please tell me more about it.
A participant thinks she is doing everything wrong. What do you say to her?
- You are testing the application, not the participant, therefore the participant is not doing anything wrong.
- If you still think she is doing something wrong, you must question your preparation first - especially if this happening more often to you
- You say:
- No you are doing everything correct, the application is not working correctly.
- You are doing a great job. It shows where we have to improve our application.
- You are not the first one running into troubles here, we have to change that.
- However, it is very important that she continues so that you check your app as much as possible and improve it wherever possible.
Why not give the participant all the tasks on paper
- It is unlikely that you skip one task if you have a pile of cards and take one card after another from the pile
- On paper you may skip a task more easy
- Do not let the participant sneak into the next task
- She should concentrate on the current task
- The next task might reveal the answer to the current task
How would you classify the errors you find. Which errors do you fix first?
One possibility is to classify into 3 classes:
- Immediate (must fix, if you do not fix it people will give you bad ratings and it might happen that people won't open your app a second time because of bad user experience)
- When time (should fix, many will benefit and it still affects the rating of your app)
- When drive by (minor)
- The most urgent error must be fixed as early as possible and it must get all your attention.
- There is no use in continuing with anything else before, because if you do not fix it your app will fail
- The biggest issue is if people get stuck in your app in a way that you (as an app author/publisher)
- you do not make money
- waste people's time
- waste people's money
Here would be an additional list for classifying errors:
- If the they (majority) cannot reach a goal, it is a must fix
* If the minority cannot reach a goal, it is a should fix * If they (majority) cannot reach a primary goal conveniently, it is a must fix * If few (minority) cannot reach a primary goal conveniently, it is a should fix * If they cannot reach a secondary goal conveniently, it is a should fix * Typos, broken layout (in a way that you still can read everything correctly), this is a minor error
However, this requires some feeling if the mistake shown was really an exception or if it might happen with most of the people, since you are testing with only few people in each round.
- Get participants (for 28.11.)
- Make schedule
- Get helper to bring the next participant
- Task list & scenarios on paper
- Prototype for all test cases
- Pencil, eraser, paper, scissors, post-its
- Agreements printed
- Task list & scenarios on index cards
- Call participants day before and remind (27.11.)
- Run pilot
Time plan example
|02||Read and sign the agreement about the test and NDA is necessary|
|05|| Warm-up questions |
|10|| Tasks |
|40|| Closure |
|50|| Transition |
Open issues from Q/A
- UI guidelines for Series 40 full-touch
- UI components' demo app
- Test example video by Steve Krug
- Jacob Nielsen’s blog
- Rocket Surgery Made Easy: The Do-it-yourself Guide to Finding and Fixing Usability Problems. Steve Krug (2009). ISBN-13: 978-0321657299
- Paper Prototyping: The Fast and Easy Way to Design and Refine User Interfaces. Carolyn Snyder (2003). ISBN-13: 978-1558608702
- Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. Jeffrey Rubin and Dana Chisnell (2008). ISBN-13: 978-0470185483