Please note that as of October 24, 2014, the Nokia Developer Wiki will no longer be accepting user contributions, including new entries, edits and comments, as we begin transitioning to our new home, in the Windows Phone Development Wiki. We plan to move over the majority of the existing entries over the next few weeks. Thanks for all your past and future contributions.
Mobile Design Pattern: Extended audio functionalities
This design pattern is part of the Mobile Design Patterns series.
Mobile is a gadget used on the go, with divergent backdrop of its usage from sedentary to while traveling, from using it as just a communication device to a camera to a radio and what not. This coupled with the constraints of the hardware like limited display real estate etc, along with many processes and applications running at the same time, catching the user’s attention for critical things while not bugging them too much at the same time remains a major challenge for any usability/design expert.
There are lots of ways to get the user’s attraction from using visual cues like notifications, blinking lights, dialogs etc to haptics/touch feed etc to the use of audio notifications/commands etc. Audio is used to denote actions like tapping of the screen, incoming calls (ring-tones), messages (beep) etc. But there are still lots of use-cases/places where the extended usage of audio commands/notifications could possibly provide exciting opportunities.
Use of audio could also enhance the overall usability experience of the applications which allow this as an interaction mode, as it would be like an additional interaction medium between the user and the device.
Extended functionalities possible with audio
Some of the existing functionalities available on the mobile could be enhanced from a usability/easy of use perspective with the usage of audio commands/notifications, some of them are as under:-
Right now to indicate incoming calls, we can make use of ring tones, or in cases of silent profile the device blinks to notify the user of an incoming call. There are times where we can’t possibly look at the device because we are driving or the phone is kept away from us and is not accessible. In those cases it would be nice to know who is calling, one can possibly set group/contact ring tones to know who is calling, but this approach is cumbersome if one has lot of contacts, the other issue with this is that you would have to memorize what ring tone belongs to which contact/group etc.
During such times it might be very handy if the device could pick up the name of the calling party if it is available in the phonebook and announce it to the user, or in case of call from someone whose number is not there in the phonebook just announcing the number/anonymous could be very helpful for the user. This way the user would know who is calling without looking at the phone screen or having to remember the associated ring tone details.
In addition to the above notification mechanism, audio commands could be used to handle calls. For instance right now there is very limited functionality available through SIND (Speaker Independent Name Dialing), details can be had from Text to speech in symbian 2nd and 3rd edition phones . The usage of audio commands can be extended to handle calls, for example when you are notified of a call from someone whose call you can’t/don’t want to answer at that point in time, you can just say Hang-up or send Busy and the call is handled accordingly. This kind of call handling functionality could be very helpful in cases where you can’t possibly interact physically with the device by pressing the keys etc.
Messaging is another inbuilt application which can enhance its usability with the usage of audio. There are cases where you get a sms and don’t have the patience/time to actually read it out after pressing the view command. In those cases you could simply be notified again of who has sent you a message, in case you know it’s a message you don’t care for much say a marketing campaign message etc, you could just ask to delete the message. While in cases where you know that the message could be important after you have been told who has sent the message, you can ask the device to read the message to you.
Lot of users find it very cumbersome to actually key in the entire message, they could certainly benefit a lot from the possibility of narrating their message and the device interprets it into words and puts them into the message. Once the user is satisfied with the message or notified by the messaging application that the sms length has been reached, they can ask to verify the message, after which they can either decide to save the messages in the drafts or send them out.
Another add on that can be built in to the messaging application could be the possibility of reading out the name of the person you want to send the message to, at which point the application picks up the name and the associated number from the phonebook and adds that to the recipient number field.
Device lock/unlock, other notifications
There are lots of times we keep the device in our pockets after having locked it and then due to the slider moving or whatever, the device ends up getting unlocked. In those cases we don’t know it has and sometimes inadvertently we end up placing calls or doing something we didn’t want to do in the first place because some or the other keys keep getting pressed. In those cases it would be very handy to be notified with an audio that the keypad has gotten unlocked, on which we can either ask the device to lock itself or decide to bring out the phone and lock it ourselves. Similarly other notifications like battery status can also be given to the user through the audio channel.
Some of the key advantages of using audio as an interaction medium between the user and the device are as under:-
Visually challenged users
It would be an absolute boon for visually challenged users who right now depend upon Braille as a communication medium. If they are able to talk to their device in the real literal sense of it, they would surely be able to make more use of the features of the phone without having to depend upon someone else to help them out.
Constrained usage scenarios
Audio interaction mechanism would also be very handy for other users in situations where they are not able to use the key/screen of the device due to some constraints like driving/being physically away from the device etc. They can both be notified of events and be able to command their phone to do some action based on those events.
However with every good feature, comes its share of problems and disadvantages as well. Some of the issues with audio interaction could be:-
Ambient noise can confuse the device
Ambient noise and external voice could end up confusing the phone into doing something which the user might not have intended it to do in the first place. For instance while driving if some bystanders say something which the phone interprets as a command it would not be a good idea.
Pattern matching issues
The device/underlying software could run into issues of pattern matching as to what should be picked as commands and what not. Lot of times we could say things which couldn’t necessarily be commands for the device but it could end up interpreting it that ways. The implementation needs to take care of such aspects.
The user could have generic settings linked to the profile or otherwise when to use audio as an interaction mode. For instance while in a meeting or something one wouldn’t want to be notified of a call, with an audio, or while driving/swimming etc they would like to make use of such functionality so that even without having to look at the device they know what is happening. The settings could be per application or the user could choose to have a set of rules/patterns/commands which could be applicable to all applications. For instance Call would mean the same to all application which allows the user to place calls, like call log, contacts, native dialer.
The usage of audio interaction could possibly be extended to other applications, both in-built and 3rd party, to allow the user greater ease of use while it comes to using those applications.
--- Added by Mayank on 24/06/2009 ---