×

Discussion Board

Page 1 of 2 12 LastLast
Results 1 to 15 of 22
  1. #1
    Regular Contributor
    Join Date
    Aug 2006
    Location
    Spain
    Posts
    63

    RecordStore bug on Series 60

    Hi all,
    I sent a comment on this a few months ago through the platform comments form, but had no response.
    I've discovered a bug on nokia Series 60 phones, like 6670, 6680 or N70 that fills up the phone memory/memory card over time, and short time in fact for certain models with less memory.
    Now I have an E70, and the bug not only persists, but it's even worse. The phone is great, but a bit buggy.
    The following code snippet detects the error:

    RecordStore rs1 = RecordStore.openRecordStore( RSNAME1, true );
    rs_initialSize = rs1.getSizeAvailable();
    rs1.addRecord( new byte[ 100 ], 0, 100 );
    rs_availableAfterSet100 = rs1.getSizeAvailable();
    rs1.setRecord( 1, new byte[ 1000 ], 0, 1000 );
    rs_availableAfterSet1000 = rs1.getSizeAvailable();
    rs1.setRecord( 1, new byte[ 10000 ], 0, 10000 );
    rs_availableAfterIncreasedSet = rs1.getSizeAvailable();
    rs1.setRecord( 1, new byte[ 1000 ], 0, 1000 );
    rs_availableAfterReducedSet = rs1.getSizeAvailable();
    rs1.deleteRecord( 1 );
    rs_availableAfterDeleteRecord = rs1.getSizeAvailable();
    rs1.closeRecordStore();

    RecordStore.deleteRecordStore( RSNAME1 );

    rs1 = RecordStore.openRecordStore( RSNAME1, true );
    rs_availableAfterRSDelete = rs1.getSizeAvailable();
    rs1.closeRecordStore();

    RecordStore.deleteRecordStore( RSNAME1 );

    hasAllocBug = rs_availableAfterIncreasedSet < rs_availableAfterReducedSet;
    if( rs_availableAfterDeleteRecord == rs_initialSize )
    {
    allocHandling = AH_FREES_ON_RECORD_DELETE;
    }
    else if( rs_availableAfterRSDelete == rs_initialSize )
    {
    allocHandling = AH_FREES_ON_RS_DELETE;
    }
    else
    {
    allocHandling = AH_BOGUS;
    }

    This is part of a class that we have made to test device capabilities.
    N70, 6680 or 6670 have the alloc bug, but at least the free up space when recordstore is deleted.
    E70, N80 have the same bug, but deleting the recordstore not only does not free space, but it takes another 4Kb.

    Different models give different numbers, one would expect that increasing the size of a record from 1000 bytes to 10000 byes would eat up 9000 bytes. In fact it can be more or less depending on the cluster size used, but on this models it takes up to 16Kb.
    When the size is reduced again to 1000 most phones wont recover space, but will leave the allocated space to that record anyway, and it will only be recovered when the recordstore is removed.
    6680 will loose a couple of Kb every time you repeat the grow/shrink operation, but will recover the space when the recordstore is delete.
    E70 will loose 24Kb each time you execute the code above.

    Sample output from my E70:

    description free space difference comments
    initial space 58.298.368
    add 100 byte record 58.298.368 0 fits into allocated space
    grow it to 1.000 byte 58.296.320 2.048 takes 2Kb
    grow it to 10.000 byte 58.286.080 10.240 takes 10kb
    shrink it to 1.000 byte 58.284.032 2.048 takes another 2Kb, instead of using some of the already 10kb + 2kb allocated previously, here's the bug
    remove record 58.284.032 0 packing space can be costly, so we understand this
    remove recordstore 58.279.936 4.096 E70 looses another 4Kb, instead of recovering the loosed space

    I would like that somebody at nokia could confirm the bug, and tell me if they plan to fix it on next firmware releases.

    Thanks,

    Narciso Cerezo
    Elondra S.L.
    http://www.basemovil.com

  2. #2
    Regular Contributor
    Join Date
    Aug 2006
    Location
    Spain
    Posts
    63

    Re: RecordStore bug on Series 60

    Hi,
    I'm really afraid of having no response at all. I understand that most users of this forum will be developing games, and that's ok of course, but they will not likely be affected by a bug like the one described.
    We're affected, because we write business applications the run with our j2me database engine. Our database engine handles hundreds of tables with thousands of rows, that is, multi-megabyte databases with frequent updates.
    The shame is that nobody at Nokia will make even a quick response saying: "hey! we're working on it" or even "we'll never fix that bug", that'll be better than nothing.
    And I really wonder what Nokia means with "business" phones, like the new E series. Do you mean just email or a better web browser? That's not business, and will not take PDAs out from the corporate market. Business is real applications that work off-line but take the most out of the connection capabilities of mobile phones.
    If I get no response in week or so, when we'll be starting our advertising campaign, I suppose that we'll be better off recomending SonyEricsson phones to our customers. They're almost as capable, and seem to have faster screen and correct recordstores handling.

    Thanks in advance.

    Narciso Cerezo
    CEO Elondra S.L.
    http://www.basemovil.com

  3. #3
    Regular Contributor
    Join Date
    Sep 2003
    Location
    Finland
    Posts
    99

    Re: RecordStore bug on Series 60

    Hi,
    First of all, multimegabyte, hundreds of "recordStores", thousands of "records" and mobile devices just don't belong to same sentence.

    The bug you are referring to is actually a feature. RMS implementations, at least the once I've ran into (Nokia, Siemens, Alcatel, SonyEricsson), never actually free the memory nor the record id you delete. You may use the deleted ids again with setRecord(id)-method, but then the data length should be the same as the deleted data's.To get the used space back you need to recreate the record store.

    For bigger data storage with all you need functionality, you might use jsr-75's FileConnection API and implement the data storage yourself. Be aware, it won't be fast with plenty of data

    -Rooster

    BTW. Threatening never makes any good in public forums

  4. #4
    Regular Contributor
    Join Date
    Aug 2006
    Location
    Spain
    Posts
    63

    Smile Re: RecordStore bug on Series 60

    Hi rooster13,
    First of all, let me tell you that our database engine handles efficently more than 10.000 rows in a single table, with full databases of 5 or 6Mb. And as the engine has no built-in limit, we're trying even bigger databases. The search times, even with full-text-like searches, is always in the order of milliseconds. And it runs in J2ME devices. So the surely belong to the same sentence.
    I know what I'm talking about, maybe I'm not able to translate that into English in a fully understandable way.
    If you read carefully my message, you will understand that it is a bug of Series60 2nd edition that is worse on 3rd edition.
    I KNOW that all RS implemetations won't free the space of a deleted record, it's easy to understand as it would imply cpu-intensive work. What I'm describing is not that, but the fact that modifiying a record making it biggger leads to loosing space, and mofiying it back to a smaller size implies more space loosing. And in 3rd edition, deleting the recordstore does not free space and eats up another 4Kb. THAT is the bug.
    If you set a record to 1Kb it really occupies 4Kb, for example, that's understandable because it neads some headers and allocates some extra space. If you set it to 10Kb, it should take 8Kb more, or even 12Kb, but it takes more. The same when you set it to 100Kb. But the funniest thing is that when you set it back to 4Kb, for example, one would expect that no space would be allocated, as it already takes 100Kb, but it discards the 100Kb and takes another 4Kb. That is surely a BUG.
    In fact, it does not happen on other phones, such as SonyEricsson K700i, for example. The K700i does not recover space when removing or making smaller a record, but it uses the space in a understandable way and does not loose space.
    You just need to run the code snippet of my first message on different phones.
    The fact, is that we've managed the "space loosing" on 2nd edition phones by removing the recordstores with most changes and asking the server to send them again, but that won't work on 3rd edition. The only way to free space on 3rd edition is to uninstall the application.

    BTW. I'm not trying to threaten anybody, I'm just trying to attract some attention from Nokia in order to have a response for this problem. And I don't think that Nokia will feel any fear from a small company like ours :-) Take it easy.

  5. #5
    Regular Contributor
    Join Date
    Aug 2006
    Location
    Spain
    Posts
    63

    Re: RecordStore bug on Series 60

    I've read again the thread, and I still don't think it sounds as "threat", but if it sounds like that to somebody... well, it was not my intention. I've seen other users of this forum complain of having no response, and that made Nokia to reply and try to give an answer. They weren't accused of threatening.
    Please, could somebody tell me what's the way to communicate a bug?

    Threatening... I still don't understand it.

  6. #6
    Regular Contributor
    Join Date
    Sep 2003
    Location
    Finland
    Posts
    99

    Re: RecordStore bug on Series 60

    Out of curiosity, are you really saying you utilize hundreds of recordstores w/ 10000 records on j2me devices in a single midlet? In other words is your database engine written in j2me?
    If so, I'd be really interested to get to know some idea how, I've seen that even with small record sizes, let's say 200B/each, you have 1000 records and take enum from the store, it takes atleast few seconds (depends on the device of course). Also opening-closing actions slow down when RSs are getting very big.

    By threatening I was just referring to your way of mentioning: "If I get no response in week or so..."

  7. #7
    Registered User
    Join Date
    Jan 2006
    Posts
    42

    Re: RecordStore bug on Series 60

    Just to confirm that I use RS for collecting GPS records, and work with RS of 10 to 100000 records without a problem ... i didnt calculate space using ... becouse I delete RS after storing data to text file. Record is from 50 to 120 bytes long plus headers.
    I hope you know rooster13 that RS use mmc card for storage space ... in my case I have planty of space (1 GB) so way not to store large database on mobile phon ?

  8. #8
    Regular Contributor
    Join Date
    Aug 2006
    Location
    Spain
    Posts
    63

    Re: RecordStore bug on Series 60

    rooster13,
    Yes our engine is pure java, pure j2me. Using the standard recordstores as the underlying mechanism we can store thousands of records in a single recordstore.
    For example, our first commercial application of this technology is for sales force management. The database used by this application has around 30 tables, for customers, products and so on. We've tested the application with different database sizes, the biggest for the moment is 10.000 products and 3.000 customers, but we're testing with even more.
    We can make direct searches, for example, customer with code 10301. That takes around 70ms, depending on the device, of course. We can also make full-text-like searches, like "gre ap", that will match "box with green apples" or "green box with apetizers", that take a little more, usually around 150ms, but still are very usable.
    We've had to deal with the things you comment, like enums, opening and closing, the differences between devices, and so on. It has taken around two years of development...
    About the "If i get no response..." it's simply that our customers demand a solution for their series60 phones, and if we get no response, we have to make a decission about wich phones we recommend. I really ment no threatening
    Feel free to write me if you have something in mind: narciso at elondra dot com.

  9. #9
    Regular Contributor
    Join Date
    Aug 2006
    Posts
    307

    Re: RecordStore bug on Series 60

    Hi, ncerezo2

    I'm afraid that you cannot rely on RMS sizes at all. The reason is that each vendor is using its own internal structures to provide access to the records. These structures are requiring some RMS space and usually are not optimized at all. You also probably know very well that record ID is not an index, so "deleting" or "updating" record can be something really different from the firmware's point of view.

    The results you've got would be much understandable if we would be able to see how much memory the RMS system took for its own needs (I mean internal structures), but we will never see such things (according to RMS specifications).

  10. #10
    Regular Contributor
    Join Date
    Aug 2006
    Location
    Spain
    Posts
    63

    Re: RecordStore bug on Series 60

    Hi axs,
    I think i will provide the tests results on different devices to give a better understanding of the nature of the problem.
    As I said, and as you say, each vendor and even each device uses it's own structures to store rms data. This means that the space allocated varies greatly, depending on the structures used and the "cluster" size used.
    But the code snippet I provided is independent of this structures, as it simply checks that the space is consistent.
    Even if a rms structure is not optimized at all, it would seem logical to think that if it has 100Kb already allocated for a record, reducing it's size to 4Kb won't eat up more space, I think that this can not even be considered an optimization. But the fact is that series60 2nd edition and 3rd edtion allocate extra space for this operation. And 3rd edition does not free the space even after deleting a recordstore. It would also seem that this is not logical since less advanced 2nd edtion devices free the space.
    Nevertheless, 3rd edition are really buggy by now, not just with this issue. The email and web browser are great, but usually leak memory until you have to power off the phone to continue operation, for example. Applications hang and exit without notification and with no predictable pattern. We've seen this on E70 and N80.
    I like my E70, I think is a great phone, and Nokia phones in general behave better than others, are more powerful. We've managed the bug with the 2nd edition, but we can't do anything with 3rd edition as the only way to free space is uninstalling the application.

    But, after all, all I want is a simple answer from Nokia saying what is this bug. If it's not a bug, but the way the phones handle information, at least we wan't to know it. That's all.

  11. #11
    Regular Contributor
    Join Date
    Mar 2006
    Posts
    61

    Re: RecordStore bug on Series 60

    Hi
    We have been testing now this problem in S60 3rd Edition devices and there seem to be some problems in freeing memory. It seems, that when the size of a record is decreased, memory is not freed correctly. Also, memory is not released, when a record is deleted. However, we were not able to reproduce the problem in case of deleting a RecordStore. Based on our testing, memory is freed correctly, when a RecordStore is deleted.

    Now our Java R&D people are aware of this problem and it will be probably corrected in the coming S60 releases.
    Best Regards
    Ali

  12. #12
    Regular Contributor
    Join Date
    Aug 2006
    Location
    Spain
    Posts
    63

    Smile Re: RecordStore bug on Series 60

    Hi mostakhd,

    Thank you very much for your response. The code snippet is part of a series of check that our applications performs on the phone the first time the application starts, so it can decide how to behave in certain situations acording to the speed of the device or the presence of this bug, for example.
    We've checked again the results of this test from the application, and it seems that when the tests are first performed (there're no recordstores presents at that time) the space is freed correctly when the recordstore is deleted. But after the database has been downloaded (it has about 70 recordstores) if we force the check again then it fails freeing the space. It may fail after a certain number of recordstores is present, or maybe when the total size of the rms exceeds certain quantity. We're not sure now, but we'll check it further and post here any results.

    By the way, we discovered the bug on Series 60 2nd edition (6680, N70), and afterwards we checked that it was also present on 3rd edition (N80, E70).

    Thanks again.

  13. #13
    Regular Contributor
    Join Date
    Sep 2003
    Location
    Finland
    Posts
    99

    Re: RecordStore bug on Series 60

    Quote Originally Posted by dpenezic
    Just to confirm that I use RS for collecting GPS records, and work with RS of 10 to 100000 records without a problem ... i didnt calculate space using ... becouse I delete RS after storing data to text file. Record is from 50 to 120 bytes long plus headers.
    I hope you know rooster13 that RS use mmc card for storage space ... in my case I have planty of space (1 GB) so way not to store large database on mobile phon ?
    Yeah, of course it does use the memory available on the device, most likely depending on where you actually install the software (phone or mem card) But with most of the devices there is a limit to size of the store.
    Just wondering.. I tried with N6280 how fast it actually is to get the enumeration from RS, with 4640 records (some limit?!?) size only few bytes each. With RecordStore.enumerateRecords(null,null,false) it took 320ms to get the enum. So what the heck I'm doing wrong? if some folks can fit text search in the same time frame with more records. The enum couldn't be simpler than that?
    Last edited by rooster13; 2006-09-15 at 16:56.

  14. #14
    Regular Contributor
    Join Date
    Aug 2006
    Location
    Spain
    Posts
    63

    Re: RecordStore bug on Series 60

    rooster13,

    As you say, most devices impose a limit either on the available memory for RMS, for a single Midlet, or for a single recordstore, or all of them at the same time.
    Nokia S60 do not have such a limit. Neither does SonyEricsson K700i and other recent mid-range phones from SE.
    Nokia S40 has limits on recordstore size, ranging from 32Kb for older phones to 262Kb for newer ones, like 6280 or the likes.
    And for the enumerations... you are doing right, but they're very slow. RMS is not designed for handling so many data, but just for hi-scores. Modern phones have the power to do more and they can handle it, but the mechanism of the RMS to wich they're bound are very tight and don't leave room for better solutions.
    I can't tell you more by now. You'll understand, we've invested a lot of time and money and we need some ROI. We pretend, in the near future, to open our engine and tools to 3rd parties, so they can focus on developing business applications and forget about database, synchronization and infrastructure.

  15. #15
    Registered User
    Join Date
    Oct 2007
    Posts
    7

    Re: RecordStore bug on Series 60

    hi,

    i see that this thread is fairly old, and some of the bugs may already have been fixed.

    however, a general question is, if i update my variable-sized record (say, the size varies from 100 to around 3000 bytes) frequently, is it generally advisable never to overwrite the record with less data than it held before.

    codewise, is something like that bringing any good?:

    int datasize = ... // new size of my record
    int oldsize = store.getRecordSize(id); // old size of my record
    byte buf[] = new byte[Math.max(oldsize, datasize)];
    // write datasize bytes to buf
    store.setRecord(id, buf, 0, buf.length); // may increase the size, but never decrase it

    thanks
    greg

Similar Threads

  1. bug in Canvas, Series 60?
    By lukaszem in forum Mobile Java General
    Replies: 8
    Last Post: 2005-12-14, 21:40
  2. Bug report for Series 80 beta 2 SDK
    By svdwal in forum Symbian Tools & SDKs
    Replies: 0
    Last Post: 2004-07-29, 09:20
  3. Series 60 Concept Emulator (SDK Beta 0.2 Linux) not working
    By mattbee in forum Mobile Java Tools & SDKs
    Replies: 1
    Last Post: 2003-06-10, 11:43
  4. Series 60Series 60 MIDP Concept SDK Beta 0.2 Linux bug?
    By kauppi in forum Mobile Java Tools & SDKs
    Replies: 3
    Last Post: 2003-04-07, 09:05

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
×