Please note that as of October 24, 2014, the Nokia Developer Wiki will no longer be accepting user contributions, including new entries, edits and comments, as we begin transitioning to our new home, in the Windows Phone Development Wiki. We plan to move over the majority of the existing entries. Thanks for all your past and future contributions.

How to run a successful beta test on Windows Phone

From Wiki
Jump to: navigation, search
Featured Article
03 Nov

This article provides tips and guidance for how to run a successful beta test on Windows Phone.

WP Metro Icon Bug.png
WP Metro Icon WP8.png
SignpostIcon WP7 70px.png
Article Metadata
Created: Mendzapp Ltd. (25 Oct 2013)
Last edited: hamishwillee (05 Nov 2013)



Consumers are extremely effective testers, and increasingly unforgiving of poorly behaving apps. As a negative rating in the early stages can have a lasting effect on the success of an app (even if defects are later fixed), effective beta testing is essential. Beta testing provides the same "real world" testing, but by a friendly audience who are excited to help improve the app.

Windows Phone allows you to beta test an app by distributing a customer-ready release build to a limited number of users for a limited time. The reasons and mechanisms for beta testing are well documented in Beta testing your app and in-app products (Dev Center): in essence you supply a list of Microsoft Account IDs and their associated email addresses, DevCenter gives the app a basic test and sends you a link which your beta test group can use to download the app.

What is not so well documented is how to run a successful beta test on Windows Phone. This article provides a number of tips and tools that can help.

Note.pngNote: The advice was compiled while beta testing Foundbite for Windows Phone 8 using around 800 users. While this is nowhere near the number of beta testers for apps like 6tag, they have provided just as much value.

How to get people interested

Any user with a Microsoft Account and a Windows Phone device is a potential beta tester. Ask for help in any community you're part of, including developer communities, special interest groups, workshops. Any beta testing is better than nothing, so don't forget your friends and family.

There are a number of "media" organisations who have great social media channels for Windows Phone app users. It is worth reaching out to them and seeing if they will cover the launch of a beta:

There are also some local communities that can help find beta testers - for example can help find beta testers for German language users.

If you're working through an app incubator program (like AppCampus) they may be able to help promote your beta test.

Tip.pngTip: WMPoweruser requires you to draft up a post on your own but is definitely worth the effort as many Windows Phone users read it (especially in emerging markets). For the Foundbite beta we were lucky enough to get a great bit of coverage from Rich Edmonds at the start and a month into the beta.

Get support from WP media channels

How to deal with signups

For very small-scale beta test developers can use any number of tools, ranging from having to email the developers directly, Google Docs, tweeting or posting in a forum.

These approaches do not scale well because of the need to maintain the list of people interested in the app separately of the email solution (resulting in need for copy-pasting etc). For larger scale testing, consider using a CRM or other system to manage large numbers of users and emails.

At time of write, we only cover one: MailChimp.


MailChimp is a emailing marketing service. It provides a sign-up form with which you can collect information needed from potential beta testers including name, device type, Microsoft Account ID etc. It gives you a list of your signups which you can email to, and also send to Microsoft Dev Center for adding to the beta.

Note.pngNote: MailChimp is free provided you send less than 12,000 emails a month - more than enough for most Beta tests]].

MailChimp also allows you to segment the list of testers. Segmenting by signup date or "batch" allows you to manage adding them to the list in DevCenter, and to just send the signup link to new testers (instead of everyone).

Tip.pngTip: Getting users from MailChimp into the Windows Phone developer centre
The Windows Phone developer centre requires that beta users be added to a semi-colon delimited list. This can be a tiresome if there is a significant number of users. To make this process easier I made a little Windows Forms app that takes in a CSV form (which you can export from MailChimp or Google Docs), verifies all of the emails and then pastes them to the clipboard separated by semi-colons. Download and find out more about it here.

Make it easy for testers to provide feedback

Making it easy for testers to provide feedback will help in getting the best bug reports and enhancement suggestions. One approach is to add a "feedback" link to all app menus so that there is never more than two clicks to ask for help. If "contextual" information can be provided by the app (for example the current page, stack, or crash information), this should be pre-seeded in the email.

Foundbite beta test app - with feedback menu

Tip.pngTip: There are various tools for managing feedback emails. Foundbite used a (free for a single user) Uservoice account where I can see all the support requests and also provide users with a forum where users can vote for features and prioritise them for development


The tips/techniques described in this article were collated from experience beta testing Foundbite for Windows Phone 8 with around 800 users. Please feel free to enhance this article with other tips, tools and suggestions.

Note.pngNote: This article was originally posted as Running a Beta on Windows Phone (blog). The author has kindly posted it here and we've modified it for wiki format.

This page was last modified on 5 November 2013, at 04:52.
253 page views in the last 30 days.