How to think "testing" when starting your sw project
This a quite difficult topic to start with, but with a few projects and experience from them, there should be a routine to follow when starting the design for a new S60 3rd edition application. Here are but a few pointers:
- Early on think what caps will be needed? Any manufacturer / platform caps? You can start this process by the different functional components you are going to implement. Any file operations, any networking, any messaging features, etc. If you can identify any areas that would require sensitive capabilities, it would help you later on to make a small study if any of these features could be implemented by working around those sensitive capabilities. If not, prepare to request a development-phase certificate for your app (so that you can test your app on a real device). Prepare also for thorough Symbian Signed & Symbian Signed for Nokia testing (by an external test house).
- Use external consultancy services to help with testing related issues <link to consultancy guide>
- Design and write you sw components to support unit testing tools. This will help you with unit tests and possible regression testing. Write your code from the testing point of view
- Do extensive testing on the emulator, run with platform security enforcement enabled and see what capabilities are reported to be missing from your project.
- Early on, define your components (if any) that will need sensitive capabilities. Design them to be in their separate (embedded) SIS packages. These separate packages should be developed and tested first. Reason behind this is that every time you submit something to Symbian Signed for Nokia testing there will be a cost to you. By first developing and testing the part that really needs to go through the Symbian Signed for Nokia and signing it, you can limit the testing cost of the remainder of your project. Another view to this to make the point: had you had your sensitive components in the same SIS package with your UI for an example – now lets assume your critical components have been through Symbian Signed for Nokia and are “ready” – still every consequent change you make to your UI (you actually make a change to the whole signed SIS package) requires you to re-submit the whole package to Symbian Signed for Nokia, thus bringing in more costs – and delay from testing. These costs can be reduced by compartmentalizing the parts that really need to be certified. Then any version updates you make need not go through Symbian Signed testing IF you have not touched the Symbian Signed tested, embedded SIS contents. Only problem here is that you will need to identify and design these parts early on.
- Use the Carbide extension (not yet published) Capability Scanner to see the delta between your defined capabilities and the actual capability need by API calls. (only submit those caps that your project uses, do not fill in every capability “just in case”).
- Do extensive testing on your software before submitting to the final test house testing. Go trough both Symbian Signed and Symbian Signed for Nokia test criteria. Identify any inconsistencies with the test criteria and correct your code accordingly – OR, if your application functionality requires to “go around” some of the test cases, you can now write a waiver request for particular test cases. Submit these waivers with your application package. This will reduce the amount of correspondence needed between you, the test house, and Nokia Developer. Think carefully what you want to be waived (practicality and actual necessity); a waiver for disabling a device to make emergency calls will not be accepted under any circumstance ;)