×
Namespaces

Variants
Actions
(Difference between revisions)

Measuring usability of a Web Runtime widget

From Nokia Developer Wiki
Jump to: navigation, search
jappit (Talk | contribs)
(This article shows how to programmatically measure usability of a Web Runtime widget)
 
hamishwillee (Talk | contribs)
m (Hamishwillee - Tidy wiki text)
 
(5 intermediate revisions by 2 users not shown)
Line 1: Line 1:
[[Category:Code Examples]][[Category:Web Runtime (WRT)]][[Category:How To]][[Category:Usability]]
+
[[Category:Code Examples]][[Category:Symbian Web Runtime]][[Category:How To]][[Category:Usability]]
This article shows a '''practical technique to measure the usability of a Web Runtime widget'''.
+
{{ArticleMetaData <!-- v1.1 -->
 +
|sourcecode= [[Media:MeasuringUsabilityWidget.zip]]
 +
|installfile= <!-- Link to installation file (e.g. [[Media:The Installation File.sis]]) -->
 +
|devices= <!-- Devices tested against - e.g. ''devices=Nokia 6131 NFC, Nokia C7-00'') -->
 +
|sdk= <!-- SDK(s) built and tested against (e.g. [http://linktosdkdownload/ Qt SDK 1.1.4]) -->
 +
|platform= <!-- Compatible platforms - e.g. Symbian^1 and later, Qt 4.6 and later -->
 +
|devicecompatability= <!-- Compatible devices e.g.: All* (must have internal GPS) -->
 +
|dependencies= <!-- Any other/external dependencies e.g.: Google Maps Api v1.0 -->
 +
|signing= <!-- Signing requirements - empty or one of: Self-Signed, DevCert, Manufacturer -->
 +
|capabilities= <!-- Capabilities required by the article/code example (e.g. Location, NetworkServices. -->
 +
|keywords= <!-- APIs, classes and methods (e.g. QSystemScreenSaver, QList, CBase -->
 +
|id= <!-- Article Id (Knowledge base articles only) -->
 +
|language= <!-- Language category code for non-English topics - e.g. Lang-Chinese -->
 +
|translated-by= <!-- [[User:XXXX]] -->
 +
|translated-from-title= <!-- Title only -->
 +
|translated-from-id= <!-- Id of translated revision -->
 +
|review-by= <!-- After re-review: [[User:username]] -->
 +
|review-timestamp= <!-- After re-review: YYYYMMDD -->
 +
|update-by= <!-- After significant update: [[User:username]]-->
 +
|update-timestamp= <!-- After significant update: YYYYMMDD -->
 +
|creationdate= 20090622
 +
|author= [[User:Jappit]]
 +
}}
 +
 
 +
 
 +
This article shows a practical technique to measure the usability of a Web Runtime widget.
  
 
==Description==
 
==Description==
Usability is a property depending on multiple factors, that gives an overall measure of how an application can be used to achieve specific goals. These factors include '''effectiveness''' (degree of accuracy of the reached goal, compared to the planned one), '''efficiency''' (time and steps required to reach the goal) and '''user satisfaction''' (an overall measure of the user feelings about the application's usage).
+
Usability is a property depending on multiple factors, that gives an overall measure of how an application can be used to achieve specific goals. These factors include effectiveness (degree of accuracy of the reached goal, compared to the planned one), efficiency (time and steps required to reach the goal) and user satisfaction (an overall measure of the user feelings about the application's usage).
  
'''Several methods are available to measure and improve the usability''' of a mobile applications, and some of them are listen on Forum Nokia Library: [http://library.forum.nokia.com/index.jsp?topic=/Design_and_User_Experience_Library/GUID-27839792-942C-4A77-9D8F-9BDE446140C7.html http://library.forum.nokia.com/index.jsp?topic=/Design_and_User_Experience_Library/GUID-27839792-942C-4A77-9D8F-9BDE446140C7.html].
+
Several methods are available to measure and improve the usability of a mobile applications, and some of them are listen on Nokia Developer Library: [http://library.developer.nokia.com/index.jsp?topic=/Design_and_User_Experience_Library/GUID-27839792-942C-4A77-9D8F-9BDE446140C7.html http://library.developer.nokia.com/index.jsp?topic=/Design_and_User_Experience_Library/GUID-27839792-942C-4A77-9D8F-9BDE446140C7.html].
  
 
===End users testing===
 
===End users testing===
One of the best and most useful methods to have an actual measure of the usability of an application, is to let '''final users directly test the application'''. This method allows to '''highlight issues and problems''' that final users could encounter with an application, while trying to reach their goals.
+
One of the best and most useful methods to have an actual measure of the usability of an application, is to let final users directly test the application. This method allows to highlight issues and problems that final users could encounter with an application, while trying to reach their goals.
  
 
==Measuring efficiency==
 
==Measuring efficiency==
 
This approach allows to have an explicit measure of the efficiency of a Web Runtime widget, related to a specific goal, by taking into account these factors:
 
This approach allows to have an explicit measure of the efficiency of a Web Runtime widget, related to a specific goal, by taking into account these factors:
* the '''number of clicks performed/required'''
+
* the number of clicks performed/required
* and the ''''time spent to reach a goal'''
+
* and the 'time spent to reach a goal
This technique allows to '''measure efficiency while users are actually using the widget, without interfering with the normal widget's behavior''', by silently monitoring clicks and time spent. For this reason, this technique can be also '''integrated in widgets released to the public''', in order to have a '''continuous measure of usability in large sets of users''' (of course, in this case, '''privacy issues''' must be taken into account, and '''explicit permission''' to the user must be asked).
+
This technique allows to measure efficiency while users are actually using the widget, without interfering with the normal widget's behavior, by silently monitoring clicks and time spent. For this reason, this technique can be also integrated in widgets released to the public, in order to have a continuous measure of usability in large sets of users (of course, in this case, privacy issues must be taken into account, and explicit permission to the user must be asked).
 +
 
 
===The HTML code===
 
===The HTML code===
Let's build a '''sample widget with 3 screens''' containing some clickable elements, and with a '''"goal" element placed on the third screen'''.<br/>
+
Let's build a sample widget with 3 screens containing some clickable elements, and with a "goal" element placed on the third screen.<br/>
 
[[Image:Wrt_measuringusability.png]]<br/>
 
[[Image:Wrt_measuringusability.png]]<br/>
 
<code html4strict>
 
<code html4strict>
Line 61: Line 87:
 
</code>
 
</code>
 
Things to note about the above code:
 
Things to note about the above code:
* the '''gotoScreen()''' function allows the user to go '''from the current screen to the screen that has the specified ID'''
+
* the {{Icode|gotoScreen()}} function allows the user to go from the current screen to the screen that has the specified ID
* in the last screen ('''screen_2''') the '''goal is represented by the DIV element containing the "This is the widget's goal!" text'''. This element calls the '''userGoalReached()''' function, defined later, that notifies about the goal reaching.
+
* in the last screen (screen_2) the goal is represented by the DIV element containing the "This is the widget's goal!" text. This element calls the {{Icode|userGoalReached()}} function, defined later, that notifies about the goal reaching.
 
===The JavaScript code===
 
===The JavaScript code===
 
====Number of clicks and time variables====
 
====Number of clicks and time variables====
First, '''two variables''' are defined in order to '''hold the number of clicks performed by the user, and the initial time of the usability analysis''':
+
First, two variables are defined in order to hold the number of clicks performed by the user, and the initial time of the usability analysis:
 
<code javascript>
 
<code javascript>
 
/* number of clicks performed */
 
/* number of clicks performed */
Line 73: Line 99:
 
var startTime = null;
 
var startTime = null;
 
</code>
 
</code>
 +
 
====Monitoring the user actions====
 
====Monitoring the user actions====
In order to count the performed clicks, the widget must listen to all click events, and increase the overall count (stored in the '''userClicks''' variable) for each of them. The following '''startUsabilityAnalysis()''' function:
+
In order to count the performed clicks, the widget must listen to all click events, and increase the overall count (stored in the userClicks variable) for each of them. The following startUsabilityAnalysis() function:
* '''initializes the number of clicks to zero'''
+
* initializes the number of clicks to zero
* '''stores the analysis' initial time'''
+
* stores the analysis' initial time
* '''starts listening to the click events''' performed within the widget
+
* starts listening to the click events performed within the widget
 
<code javascript>
 
<code javascript>
 
function startUsabilityAnalysis()
 
function startUsabilityAnalysis()
Line 88: Line 115:
 
}
 
}
 
</code>
 
</code>
For each click event, the '''userClick() function''' is called, and it has to '''increment the overall clicks' count''' by one:
+
For each click event, the {{Icode|userClick()}} function is called, and it has to increment the overall clicks' count by one:
 
<code javascript>
 
<code javascript>
 
function userClick()
 
function userClick()
Line 96: Line 123:
 
</code>
 
</code>
 
====Reaching the planned goal====
 
====Reaching the planned goal====
When the '''user reaches the planned goal''', the widget has to '''calculate the time spent to reach it''', and to use the measured quantities (clicks and time) in a way that can be useful for a further analysis, as:
+
When the user reaches the planned goal, the widget has to calculate the time spent to reach it, and to use the measured quantities (clicks and time) in a way that can be useful for a further analysis, as:
* if the testing users are remote, these '''values can be sent via Ajax calls'''. Multiple measures can also be performed, in different usage sessions, and '''sent in an aggregate Ajax call''' to minimize the network traffic.
+
* if the testing users are remote, these values can be sent via Ajax calls. Multiple measures can also be performed, in different usage sessions, and sent in an aggregate Ajax call to minimize the network traffic.
* if the test is local, the '''values can also be shown on screen''', so that the final users and/or the developers and designers can immediately easily read them
+
* if the test is local, the values can also be shown on screen, so that the final users and/or the developers and designers can immediately easily read them
 
In this example, the measured values are shown is an alert message, but any other approaches, as the one described above, are possible:
 
In this example, the measured values are shown is an alert message, but any other approaches, as the one described above, are possible:
 
<code javascript>
 
<code javascript>
Line 116: Line 143:
 
</code>
 
</code>
 
==Further improvements==
 
==Further improvements==
'''Improvements and adjustments''' can be done to the code presented in this article, in order to adapt it to the different needs of different widgets. Examples are:
+
Improvements and adjustments can be done to the code presented in this article, in order to adapt it to the different needs of different widgets. Examples are:
* '''counting wrong clicks''': this examples only counts the global clicks, without checking where the click was performed. In a finer analysis, also '''the information about the click targets could be useful''', in order to see where the user clicked, and use this information to modify or adjust the widget's layout
+
* counting wrong clicks: this examples only counts the global clicks, without checking where the click was performed. In a finer analysis, also the information about the click targets could be useful, in order to see where the user clicked, and use this information to modify or adjust the widget's layout
* '''defining multiple goals''': a widget could have more than a single goal for a specific action. In this case, all the goals could have the same importance, or '''a goal could be more important than the others'''. In this case, the '''goal importance should be quantified and stored''' with the other values (time and clicks) to allow further analysis
+
* defining multiple goals: a widget could have more than a single goal for a specific action. In this case, all the goals could have the same importance, or a goal could be more important than the others. In this case, the goal importance should be quantified and stored with the other values (time and clicks) to allow further analysis
* '''measuring effectiveness''': the proposed approach measures the efficiency of a widget, related to a specific goal. In order to measure effectiveness, a the completeness of the goal must also be measured (e.g.: measuring completeness of entered information).
+
* measuring effectiveness: the proposed approach measures the efficiency of a widget, related to a specific goal. In order to measure effectiveness, a the completeness of the goal must also be measured (e.g.: measuring completeness of entered information).
  
 
==Downloads==
 
==Downloads==
 
The sample download presented in this article is available for download here: [[Media:MeasuringUsabilityWidget.zip]]
 
The sample download presented in this article is available for download here: [[Media:MeasuringUsabilityWidget.zip]]

Latest revision as of 08:24, 10 February 2012

Article Metadata
Code ExampleArticle
Created: jappit (22 Jun 2009)
Last edited: hamishwillee (10 Feb 2012)


This article shows a practical technique to measure the usability of a Web Runtime widget.

Contents

[edit] Description

Usability is a property depending on multiple factors, that gives an overall measure of how an application can be used to achieve specific goals. These factors include effectiveness (degree of accuracy of the reached goal, compared to the planned one), efficiency (time and steps required to reach the goal) and user satisfaction (an overall measure of the user feelings about the application's usage).

Several methods are available to measure and improve the usability of a mobile applications, and some of them are listen on Nokia Developer Library: http://library.developer.nokia.com/index.jsp?topic=/Design_and_User_Experience_Library/GUID-27839792-942C-4A77-9D8F-9BDE446140C7.html.

[edit] End users testing

One of the best and most useful methods to have an actual measure of the usability of an application, is to let final users directly test the application. This method allows to highlight issues and problems that final users could encounter with an application, while trying to reach their goals.

[edit] Measuring efficiency

This approach allows to have an explicit measure of the efficiency of a Web Runtime widget, related to a specific goal, by taking into account these factors:

  • the number of clicks performed/required
  • and the 'time spent to reach a goal

This technique allows to measure efficiency while users are actually using the widget, without interfering with the normal widget's behavior, by silently monitoring clicks and time spent. For this reason, this technique can be also integrated in widgets released to the public, in order to have a continuous measure of usability in large sets of users (of course, in this case, privacy issues must be taken into account, and explicit permission to the user must be asked).

[edit] The HTML code

Let's build a sample widget with 3 screens containing some clickable elements, and with a "goal" element placed on the third screen.
Wrt measuringusability.png

<html>
[...]
 
</head>
<body onLoad="javascript:init();">
<div id="screen_0" class="screen">
 
<div class="wrong_input" onclick="alert('go nowhere')">
Do not click me!
</div>
 
<div class="correct_input" onclick="gotoScreen('screen_1')">
Go to next screen
</div>
 
</div>
 
<div id="screen_1" class="screen" style="display: none;">
<div class="wrong_input" onclick="alert('go nowhere')">
Do not click me!
</div>
 
<div class="wrong_input" onclick="alert('go nowhere')">
This brings you nowhere
</div>
 
<div class="correct_input" onclick="gotoScreen('screen_2')">
Go to next screen
</div>
 
</div>
 
<div id="screen_2" class="screen" style="display: none;">
<div class="correct_input" onclick="userGoalReached()">
This is the widget's goal!
</div>
 
</div>
</body>
</html>

Things to note about the above code:

  • the gotoScreen() function allows the user to go from the current screen to the screen that has the specified ID
  • in the last screen (screen_2) the goal is represented by the DIV element containing the "This is the widget's goal!" text. This element calls the userGoalReached() function, defined later, that notifies about the goal reaching.

[edit] The JavaScript code

[edit] Number of clicks and time variables

First, two variables are defined in order to hold the number of clicks performed by the user, and the initial time of the usability analysis:

/* number of clicks performed */
var userClicks = 0;
 
/* analysis initial time */
var startTime = null;

[edit] Monitoring the user actions

In order to count the performed clicks, the widget must listen to all click events, and increase the overall count (stored in the userClicks variable) for each of them. The following startUsabilityAnalysis() function:

  • initializes the number of clicks to zero
  • stores the analysis' initial time
  • starts listening to the click events performed within the widget
function startUsabilityAnalysis()
{
userClicks = 0;
 
startTime = new Date().valueOf();
 
document.addEventListener('click', userClick, false);
}

For each click event, the userClick() function is called, and it has to increment the overall clicks' count by one:

function userClick()
{
userClicks++;
}

[edit] Reaching the planned goal

When the user reaches the planned goal, the widget has to calculate the time spent to reach it, and to use the measured quantities (clicks and time) in a way that can be useful for a further analysis, as:

  • if the testing users are remote, these values can be sent via Ajax calls. Multiple measures can also be performed, in different usage sessions, and sent in an aggregate Ajax call to minimize the network traffic.
  • if the test is local, the values can also be shown on screen, so that the final users and/or the developers and designers can immediately easily read them

In this example, the measured values are shown is an alert message, but any other approaches, as the one described above, are possible:

function userGoalReached()
{
document.removeEventListener('click', userClick, false);
 
var endTime = new Date().valueOf();
 
var totalTime = endTime - startTime
 
alert(
"Performed clicks: " + userClicks + "\n" +
"Time spent: " + (totalTime / 1000) + " seconds."
);
}

[edit] Further improvements

Improvements and adjustments can be done to the code presented in this article, in order to adapt it to the different needs of different widgets. Examples are:

  • counting wrong clicks: this examples only counts the global clicks, without checking where the click was performed. In a finer analysis, also the information about the click targets could be useful, in order to see where the user clicked, and use this information to modify or adjust the widget's layout
  • defining multiple goals: a widget could have more than a single goal for a specific action. In this case, all the goals could have the same importance, or a goal could be more important than the others. In this case, the goal importance should be quantified and stored with the other values (time and clicks) to allow further analysis
  • measuring effectiveness: the proposed approach measures the efficiency of a widget, related to a specific goal. In order to measure effectiveness, a the completeness of the goal must also be measured (e.g.: measuring completeness of entered information).

[edit] Downloads

The sample download presented in this article is available for download here: Media:MeasuringUsabilityWidget.zip

This page was last modified on 10 February 2012, at 08:24.
60 page views in the last 30 days.

Was this page helpful?

Your feedback about this content is important. Let us know what you think.

 

Thank you!

We appreciate your feedback.

×