×

Discussion Board

Results 1 to 14 of 14
  1. #1
    Registered User
    Join Date
    May 2013
    Posts
    280

    [moved] Is Kxml parser a slow parser?

    I am working on s40 app and using kxml parser. The data in my UI is coming from kxml parser. I am able to do parsing fine ..But with page of 600kb it is taking 4-5 minutes for parsing 7000 records . I am storing each in a string and then in vector.

    Is kxml a slow parser because i heard other parser take just 5-6 seconds to parse even this much of data.
    What could be the reason

  2. #2
    Nokia Developer Champion
    Join Date
    Feb 2008
    Location
    Ahmedabad, Gujarat, India
    Posts
    3,852

    Re: [moved] Is Kxml parser a slow parser?

    There are two things here:

    First i dont think KXML parser is slow, its a sax based parser and its not at all slow.

    Second your 600Kb of data is taking 4-5 minutes is a huge time, so question if you reading a data from an network connection, is network connection ok , i mean i hope the connection is not slow only. If connection is ok then your parsing routine may be consuming the time !!

    Regards,

  3. #3
    Registered User
    Join Date
    May 2013
    Posts
    280

    Re: [moved] Is Kxml parser a slow parser?

    Connection is ok ....ok i am sharing my parsing routine plz tell whats making it slow
    Here is my code

    public Vector mainparsing() throws RecordStoreFullException, RecordStoreNotFoundException, RecordStoreException
    {


    XmlParser parser = null;
    ParseEvent event = null;
    String golden_date = "";
    String golden_text = "";

    Vector v1=new Vector();
    Vector v2=new Vector();


    try
    {
    String url="http://tandonindia.com/check.php?user=1";


    HttpConnection hc = (HttpConnection)Connector.open(url);
    InputStream is = hc.openInputStream();
    Reader reader = new InputStreamReader(is);
    parser = new XmlParser(reader);
    boolean xmlParsingDone = false;

    while (!xmlParsingDone)
    {


    event = parser.read();
    ParseEvent pe;
    switch (event.getType())
    {
    case Xml.START_TAG:
    // Pick up Title for display
    String tagName = event.getName();



    if ("date".equals(tagName)) //DATE PARSING
    {

    pe = parser.read();
    if(pe.getText()!=null)
    {



    golden_date=pe.getText();


    }
    else
    {

    golden_date="null"; //NULL VALUES
    }




    v1.addElement(golden_date);





    }

    if ("goldenQuote".equals(tagName)) //THOUGHT PARSING
    {


    pe = parser.read();


    if(pe.getText()!=null)
    {

    golden_text = pe.getText();


    }
    else
    {

    golden_text="null";

    }
    v2.addElement(golden_text);


    }



    break;
    case Xml.END_TAG:
    break;
    case Xml.END_DOCUMENT:
    xmlParsingDone = true;
    break;
    case Xml.TEXT:
    break;
    case Xml.WHITESPACE:
    break;
    default:

    }

    }


    System.out.println("sizeeeeeeeeeeeeeeeeeeeeee of v1--list----DATES-----------"+v1.size());
    System.out.println("sizeeeeeeeeeeeeeeeeeeeeee of v2--list-----THOUGHTS----------"+v2.size());



    finalVector.addElement(v1);
    finalVector.addElement(v2);


    }
    catch (IOException ioe){
    System.out.println("XML Parsing Error: " + ioe);
    ioe.printStackTrace();
    }finally
    {
    parser = null;
    }
    return finalVector;


    }

  4. #4
    Nokia Developer Champion
    Join Date
    Mar 2013
    Posts
    686

    Re: [moved] Is Kxml parser a slow parser?

    KXML parser is not slow, its a pull xml parser which mean it only traverse the next tag/token/text when you ask him to, either you have alot of data and then there is nothing to be done, or you can further optimize your code (string.equals can be slow, also string aggregation with + operators can be slow so try and replace those with StringBuilder/StringBuffer i forgot which one is available in j2me) also your system.out.println are very slow if they are called in a look with the string aggregation +
    Also one last thing..... i looked over your code and i dont actually see you USE KXML parser ...... you are calling new XmlParser(Reader) which isnt KXmlParser and also you call parser.read which KXmlParser doesnt have such public method.

  5. #5
    Registered User
    Join Date
    May 2013
    Posts
    280

    Re: [moved] Is Kxml parser a slow parser?

    ok so is my code alright for parsing below by some optimizations u told?

  6. #6
    Registered User
    Join Date
    May 2008
    Location
    Surat Thani
    Posts
    260

    Re: [moved] Is Kxml parser a slow parser?

    It's easy to check whether the KXML parser is the culprit. Just remove everything from your loop except end document handling:
    Code:
    while (!xmlParsingDone)
    {
    
    event = parser.read();
    
    switch (event.getType())
    {
    case Xml.END_DOCUMENT:
    xmlParsingDone = true;
    break;
    
    }
    }
    and see how long the parsing takes then. If, after this experiment, you see that the parsing is really the thing that is causing the slowness, you could see if it helps to implement your own simpler parsing with less overhead, as described in this post:
    http://developer.nokia.com/Community...l=1#post922574

    Another thing that might be at least a partial cause of the slowness is that you're adding a huge amount data to Vectors without giving a hint about how much data you will be adding. Vector is probably internally implemented as an array of objects, and when it runs out of space in the current array, it allocates another bigger one and copies the existing data there. If you know the amount of items you will add, you can avoid these reallocations by giving the needed capacity as an argument to the Vector constructor, or calling ensureCapacity.

    A general comment about the XML parsing loop, it would be better to first collect all data for an item, and then at the item end tag validate the data and if it's ok and store all the data for the item at the same time. Now you're storing stuff in two different vectors at different times, and the whole thing will break if you get one item without date or without goldenQuote (or date or goldenQuote without an item).

  7. #7
    Registered User
    Join Date
    May 2013
    Posts
    280

    Re: [moved] Is Kxml parser a slow parser?

    I checked everything. I guess my data is too much which is taking lot of time. Since rms file for device is different which is formed on running the app on device , I am stuck that how can i reduce time as user will not be waiting for that much of time(moreover my data will be increasing day by day).
    Is there any solution for this?

    If there is a way to write data in files or anything like that(so that i can send that file directly to device since i cnt do same with rms) which can save time plz provide ways how to do that .
    Last edited by Manpreet_Singh; 2013-09-19 at 08:18.

  8. #8
    Nokia Developer Champion
    Join Date
    Feb 2009
    Location
    Noida, India
    Posts
    3,087

    Re: [moved] Is Kxml parser a slow parser?

    Quote Originally Posted by Manpreet_Singh View Post
    I checked everything. I guess my data is too much which is taking lot of time. Since rms file for device is different which is formed on running the app on device , I am stuck that how can i reduce time as user will not be waiting for that much of time(moreover my data will be increasing day by day).
    Is there any solution for this?

    If there is a way to write data in files or anything like that(so that i can send that file directly to device since i cnt do same with rms) which can save time plz provide ways how to do that .
    The data that you wish to have the app on the 1st launch itself - you can make it part of the application resource bundle.
    Incremental updates from server need to saved either in RMS or files on the using JSR 75 File Connection API's.
    thanks,
    ~Amitabh
    (Champion of the Month -Aug'13)
    Follow me on my blog for Innovative Mobile Apps

  9. #9
    Registered User
    Join Date
    May 2013
    Posts
    280

    Re: [moved] Is Kxml parser a slow parser?

    how to do so

  10. #10
    Nokia Developer Champion
    Join Date
    Feb 2009
    Location
    Noida, India
    Posts
    3,087

    Re: [moved] Is Kxml parser a slow parser?

    For reading your data from Txt File in the resource folder you can try - http://developer.nokia.com/Community...ine_in_Java_ME

    For reading/writing to file from j2me on the device read - http://developer.nokia.com/Community...n_API_(JSR_75)
    thanks,
    ~Amitabh
    (Champion of the Month -Aug'13)
    Follow me on my blog for Innovative Mobile Apps

  11. #11
    Nokia Developer Champion
    Join Date
    Mar 2013
    Posts
    686

    Re: [moved] Is Kxml parser a slow parser?

    Manpreet_Signh: as i've already wrote you in this thread the fault is not with the KXML parser
    and as TongMuan have suggested to you to try his code snippet and see how much time this take to run.
    Then you can think of ways to implement your parsing to make it efficient as possible.
    In general if your data is coming from Network as i can see from your code then i would suggest you dont try to read/parse it directly from the network stream but rather download it all to a temp file
    and then read/parse the inputstream from that file.

  12. #12
    Regular Contributor
    Join Date
    Mar 2005
    Location
    Hamburg, Germany
    Posts
    285

    Re: [moved] Is Kxml parser a slow parser?

    XML is format that has a lot of redundant information. This makes the data you transfer bigger. When you use an optimized format, your data can be much smaller, and therefore faster. Also xml-parsing is quite complex.
    Also your XML file has much several fields you than throw away. It would be faster if the server does not generate these information and instead creates a smaller xml file.

    But of course you need to define such a format and implement it on client and also on the server.
    Something quite standard and still much smaller is csv.

    But still several minutes is very long. Maybe the memory of the java-virtual-machine is near the memory limit, it is doing garbage collection all the time.
    You can try a file that is half as big. Is it using half the time (normal behaviour) are much less the half the time (then something is near the limit)

    @shai: I think it gets even slower when you first download it to a file. You have even more I-O Operations.

    Regards

    Karsten Meier

  13. #13
    Nokia Developer Champion
    Join Date
    Mar 2013
    Posts
    686

    Re: [moved] Is Kxml parser a slow parser?

    @internetlabor, reading small parts from the network stream (which is what being done with his code, since KXML doesnt read the entire input first) is much much more slower than first reading the entire network stream as big blocks (1k or 10k blocks) into a file and then reading the input from that file.
    File system I/O is always faster than network I/O

  14. #14
    Nokia Developer Moderator
    Join Date
    Feb 2006
    Location
    Oslo, Norway
    Posts
    28,752

    Re: [moved] Is Kxml parser a slow parser?

    While I know nothing about the JVM implementation in Asha/Series 40, it is rather doubtful that kXML's read requests and the actual TCP stream over the network would interlock on byte-level. There is buffering on the OS side, there is buffering on JVM side, and there may be buffering in the Java code too. All of them in the RAM which is fast enough, even if something is outside of the Java heap, and has to be copied there.
    The real problem is that this thread is rolling for almost 2 months by now (ok, there is a large gap in the middle), and nothing was measured so far
    - downloading 600k data
    - saving 600k data to file and/or RMS
    - loading 600k data from file and/or RMS and/or resource (as suggested in #10)
    - "parsing" 600k data from memory (throwing away everything as suggested in #6)
    - "parsing" 600k data from file and/or RMS and/or resource
    - "parsing" 600k data from network
    A day of work approximately.

Similar Threads

  1. Differneces between kxml parser and default xml parser in lwuit
    By rchadalawada in forum Mobile Java General
    Replies: 9
    Last Post: 2012-12-27, 07:48
  2. help in KXml parser..
    By buffon2009 in forum Mobile Java General
    Replies: 2
    Last Post: 2011-01-25, 07:26
  3. How to use the kxml parser
    By dhiraj.deka in forum Mobile Java General
    Replies: 7
    Last Post: 2009-11-03, 16:16
  4. kXML XML Parser versus Jsr172 API Parser
    By javame.developer in forum Mobile Java General
    Replies: 1
    Last Post: 2008-02-14, 20:23

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
×