×

Discussion Board

Results 1 to 3 of 3
  1. #1
    Registered User
    Join Date
    Jul 2011
    Posts
    12

    float number precision - Help ASAP

    i used float my number with double data type to count distance between 2 node
    the result of the count need a float number with precision more than 7

    the result is 0.999999999999712342

    there's no problem the simulator
    but when i deploy on my symbian device, the result always round to 1.000000 eventhough I display with qDebug::number(result,'g',12) but the result stiil 1 in my symbian device. I need the number can store more than 7 precision because this result to sensitive in my code
    could somebody help me?
    Last edited by blasius.neri; 2011-10-20 at 17:26.

  2. #2
    Super Contributor
    Join Date
    Jul 2008
    Posts
    2,020

    Re: float number precision - Help ASAP

    This is a bit strange, I dont really know what to post myself here.

    In the worst case, why not use a third party math library?
    The lines above are the best I have to offer.If anyone of you is of more advanced knowledge, I ask for your patience and understanding! - unknown arab poet
    http://www.tamoggemon.com - Symbian blog - Windows Phone blog
    My other blogs:
    webOS blog iPhone blog BlackBerry blog Samsung bada blog Android blog

  3. #3
    Registered User
    Join Date
    Oct 2010
    Posts
    256

    Re: float number precision - Help ASAP

    What are you trying to calculate? Some kind of new big bang theory Could the old trick be used where you multiply the components by very large number and do the mid math with bigger numbers?

Similar Threads

  1. Timer precision
    By sat-zok in forum Symbian
    Replies: 1
    Last Post: 2011-09-28, 13:31
  2. how separate a float number digits?
    By golzari_ali in forum Mobile Java General
    Replies: 3
    Last Post: 2010-10-30, 03:47
  3. Precision about n96 fire key
    By nikkoSH in forum Mobile Java General
    Replies: 3
    Last Post: 2009-03-24, 06:59
  4. Please help about calculation of float number
    By quatban in forum Mobile Java Tools & SDKs
    Replies: 1
    Last Post: 2006-12-05, 14:38
  5. Float using henson float
    By Pandaemonium in forum Mobile Java General
    Replies: 2
    Last Post: 2004-03-24, 10:40

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
×