×

Discussion Board

Results 1 to 9 of 9

Thread: Ovi Testing

  1. #1
    Super Contributor
    Join Date
    Jun 2003
    Location
    Cheshire, UK
    Posts
    7,395

    Ovi Testing

    Nokia’s QA process consists of the performance of a number of tests as well as a review of legal and content appropriateness on all items submitted by developers.
    Forgetting the legal/content review, is this testing to Java Verified standards? If I submit JV'd content, does my application still get tested? (In essence, do I have to pass JV twice??)

    If I submit content that has already passed JV, do I get no benefit from the costs I save Nokia, assuming that Nokia will not run the same tests if Babel/NSTL/whoever have already run them?

    Graham.

  2. #2
    Nokia Developer Expert
    Join Date
    Oct 2006
    Location
    Tampere
    Posts
    62

    Re: Ovi Testing

    The Ovi Q&A is not the same as Java Verified - or for that matter - Symbian Signed testing.
    However there is overlap, and the issue of application undergoing duplicate test criteria is one that Nokia is well aware of.

    I do expect that the issue will change - and that the process is made more simple. Forum Nokia and Ovi are looking at a number of ways that will improve and streamline the process.

  3. #3
    Super Contributor
    Join Date
    Jun 2003
    Location
    Cheshire, UK
    Posts
    7,395

    Re: Ovi Testing

    "For Java application developers, the Java Verified program should be the last step before taking their application to the market. For the operators and application resellers, the Java Verified program should be the first requirement when evaluating a Java application," said Lee Epting, vice president, Forum Nokia. "Nokia will recommend the Java Verified program throughout Forum Nokia, our 1.3 million member developer community, to assure the best possible visibility to developers."
    Well, it seems that Java Verified is not my last step, and is not a requirement for Nokia as an application reseller.

    So, what are the requirements? Where can I find Nokia's QA test cases? How do I know what tests I need to pass?

    Cheers,
    Graham.

  4. #4
    Super Contributor
    Join Date
    Dec 2005
    Location
    England,UK
    Posts
    1,600

    Re: Ovi Testing

    Quote Originally Posted by grahamhughes View Post
    Well, it seems that Java Verified is not my last step, and is not a requirement for Nokia as an application reseller.

    So, what are the requirements? Where can I find Nokia's QA test cases? How do I know what tests I need to pass?

    Cheers,
    Graham.
    Hi Graham,
    Java Verified have released the Unified Testing Criteria for Java(TM) Technology-based
    Applications for Mobile Devices


    This is the new testing policy has come into place now, as the store,
    ATOW closed, check out the JV website Java Verified Documents

    Everything is done by "FAX!",because they need signatures.

    Heres the Java Verified FAQ

    Good Luck,
    Jim
    Last edited by jimgilmour1; 2009-05-07 at 14:47.

  5. #5
    Super Contributor
    Join Date
    Jun 2003
    Location
    Cheshire, UK
    Posts
    7,395

    Re: Ovi Testing

    Thanks Jim. Yes, I've been working with JV for the last five years, so I'm very familiar with its requirements.

    However, this doesn't appear to help me with Ovi:

    Quote Originally Posted by sellis View Post
    The Ovi Q&A is not the same as Java Verified - or for that matter - Symbian Signed testing.
    Ovi testing is, apparently, not Java Verified. It seems I do not have to put my application through JV at all, before submission to Ovi (contrary to what many of us believed earlier).

    To what specification do Nokia test Ovi submissions?

  6. #6
    Super Contributor
    Join Date
    Dec 2005
    Location
    England,UK
    Posts
    1,600

    Re: Ovi Testing

    Quote Originally Posted by grahamhughes View Post
    Thanks Jim. Yes, I've been working with JV for the last five years, so I'm very familiar with its requirements.

    However, this doesn't appear to help me with Ovi:



    Ovi testing is, apparently, not Java Verified. It seems I do not have to put my application through JV at all, before submission to Ovi (contrary to what many of us believed earlier).

    To what specification do Nokia test Ovi submissions?
    Well looking at the Publisher manual, which *DOES NOT* have any confidential markers in this document from page 31


    6. QUALITY ASSURANCE (QA) PROCESS
    All content items submitted for publishing go through a quality assurance process. The main purpose of the QA is
    to ensure consumers receive quality content and application experiences through Ovi Store. We also strive to
    achieve fast publishing turnaround for developers. The best way to ensure your content is approved quickly is to
    test it thoroughly on all target devices before submitting it to Ovi Store.
    Nokia’s QA process consists of the performance of a number of tests as well as a review of legal and content
    appropriateness on all items submitted by developers. NOTE: Testing does not cover every aspect of the content
    experience or application functionalities, nor does it cover all possible phone, operator and language combinations.
    Rather, we focus on the verification of basic or core functionality. It is the responsibility of the content publisher to
    fully test, validate and sign their application content prior to submission to the Ovi Store. Do not expect Ovi
    consumer users to test your content for you. If you do, they may not buy from you again!
    After the quality assurance process is complete you will receive an email stating the status of the item in the
    system. If the QA fails, the reason and related test case will be provided by email.
    6.1 Purpose of Testing
    Nokia tests content prior to publishing to ensure consumers experience the same level of high quality across all
    content types in the Ovi Store. While it is expected that all publishers will thoroughly test their content and
    applications on all target devices before submittal, if Nokia’s QA test process identifies any problematic content, the
    32 (42) Nokia Ovi Publisher Guide, Beta version as of 2 March 2009
    publisher of that content will be notified with a summary of the test failure results. It is then the publisher’s
    responsibility to either correct the problem or change their target device or distribution selection.
    6.2 Testing criteria
    When you submit content for testing during the QA process there are factors that must be taken into account before
    submission. The publisher must check that content meets these criteria before submission.
    Generic entry criteria for testing
    · Publisher must complete the content submission form and lock the file
    · Device distribution must have at least one specific device selected Submissions will be rejected if the
    publisher automatically chooses the “Works on and can be distributed to all devices” check box
    · Country field must have at least one specific country
    · Localization or language must be stated if applicable to ensure the right application is targeted to the right
    country
    · Only Nokia billing is accepted. If the content contains any other external billing solution it will be rejected.
    Entry criteria for Symbian or Java application testing
    · Symbian application submissions (of the .sis and .sisx file type) must be pre-tested and must fulfill the
    Symbian Signed Certification requirements. Symbian signing requirements may be found at Forum Nokia
    (http://www.forum.nokia.com/Resources..._and_User_Expe
    rience/Testing/) or at the Symbian Foundation website (www.symbianfoundation.org)
    · Java application submissions must be pre-tested and fulfill Java Verified requirements Visit
    http://www.javaverified.com/ for more information on the Java Verified service
    · Java and Symbian applications must be signed appropriately, no application certificate error message will
    be allowed for content or applications on Ovi Store. For best practices on how to sign applications go to the
    Forum Nokia discussion boards at http://discussion.forum.nokia.com/fo...play.php?f=124
    · There must be “Help” and “About” available somewhere inside the application where you can find more
    information about the application that you just installed.
    · Application language use must be consistent and appropriate throughout the user experience
    All applications submitted to the Ovi Store must be able to provide proof that Symbian Signed and Java Verified
    requirements are met if requested. For more information, see http://forum.nokia.com/main/0,6566,22,00.html
    Nokia may require third party testing for applications. If the QA team receives content that frequently fails, it is the
    publisher’s responsibility to submit information to Nokia that the content works. Third party testing may be required
    as proof.
    Entry criteria for Flash Lite application testing
    · SIS files for supported S60 devices
    SIS files must be properly signed through Symbian Signed Express signed or Certified Signed
    · NFL files for supported S40 devices
    NFL files do not have to be signed
    · Flash files (SWF) are not accepted. All Flash applications must be packaged as either a SIS or an NFL file
    · Installing a SIS file to an S60 device must not install or prompt to install the Adobe Flash Lite player
    33 (42) Nokia Ovi Publisher Guide, Beta version as of 2 March 2009
    · SIS files that are created using the Adobe Mobile Packager Tool will not be accepted and will be rejected
    during the QA process. Forum Nokia offers a free web based Flash packaging service:
    http://www.forum.nokia.com/Resources...es/Flash_Lite/
    · Flash files must be published as versions 1.1, 2.0, 2.1 or 3.0. Files that are published as FL3.1 will not be
    accepted.
    · For information on device specification, please refer to
    http://www.forum.nokia.com/devices/matrix_all_1.html and select your version of Flash for a list of
    applicable devices.
    Well now you have read this, and this does not make me any wiser on the testing.

    Jim

  7. #7
    Nokia Developer Champion
    Join Date
    Apr 2005
    Location
    Boston
    Posts
    517

    Re: Ovi Testing

    Ciao,

    on another note, there is no option for developers to test content that is in the OVI Store before users downloads.

    For example, if you apply DRM, it would be nice to test the content before/after QA, to make sure everything is good to go.

    It's a typical scenario of "Smoke Test" when you release software. Testing few things to make sure everything is fine, installation, DRM and so on.

    Maybe having a "OVI Store emulator" in the developers account showing their own content, it will enable content developers to do themselves a "Smoke Test". It would be a great addition.


    Alessandro

  8. #8
    Nokia Developer Expert
    Join Date
    Jan 2008
    Location
    San Francisco
    Posts
    121

    Re: Ovi Testing

    The Ovi Store and publish to Ovi team are aware of these questions from you and other developers and we expect to have clarification very soon as to what's required for Java apps for Ovi Store in terms of signing, etc...
    [B]Bill Perry[/B]
    Sr. Services Marketing Manager, Forum Nokia
    Publishing to Ovi Store: [url]http://publish.ovi.com/[/url]
    twitter: [url]www.twitter.com/bperry[/url]
    blog: [url]www.mobileperry.com[/url]

  9. #9
    Super Contributor
    Join Date
    Jun 2003
    Location
    Cheshire, UK
    Posts
    7,395

    Re: Ovi Testing

    The fact that initial documentation suggested that Java Verified would be a requirement for Ovi submission was something I saw as a big positive for Ovi.

    If JV is not a requirement, and you will test my certified apps anyway, then I have two issues:

    1. You are implying that Java Verified is pointless.

    2. You are using the money you make from your 30% cut of my product to pay for the testing of freeware. Basically, I would be subsidising my own competition, and enabling that competition to undercut me.

    Cheers,
    Graham.

Similar Threads

  1. Replies: 7
    Last Post: 2011-02-12, 18:31
  2. Replies: 3
    Last Post: 2009-05-02, 04:32
  3. publishing to Ovi: Webinar on April 9th
    By hungster10 in forum [Closed] Publishing to Nokia Store
    Replies: 5
    Last Post: 2009-04-23, 10:17
  4. OVI attraction and easy to use
    By Carolus99 in forum [Closed] Publishing to Nokia Store
    Replies: 1
    Last Post: 2009-04-21, 18:08
  5. Replies: 0
    Last Post: 2003-10-08, 10:25

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
×