Welcome, Guest
Username: Password: Remember me
  • Page:
  • 1

TOPIC:

Array with float 08 Dec 2022 09:54 #24698

  • Horst
  • Horst's Avatar
  • Topic Author


  • Posts: 284
  • Hello
    I have a multi dim array with text and float. And i fill it with a dbf. Now when i compare the results. i have a fault.
    In fKontoSaldo and aWriteLines is the same value.
    	IF fKontoSaldo != aWriteLines [nArray][07]                   //  This is wrong !!
    	IF fKontoSaldo != Val (NTrim ( aWriteLines [nArray][07] ))    //  This works !!

    What's wrong here ?

    I hope this time my sample in the attachemt is Ok ;-)

    Horst
    Attachments:

    Please Log in or Create an account to join the conversation.

    Array with float 08 Dec 2022 10:13 #24699

    • robert
    • robert's Avatar


  • Posts: 3269
  • Horst,
    Most likely there is a rounding problem. Some floating point numbers cannot be stored 100% exactly.
    You can either add:
    SetFloatDelta(0.001)
    which tells the runtime to ignore differences smaller than 1/10 of a cent
    or compare the numbers with Round(fKontoSaldo,2)

    You will see similar problems in all apps that work with floating point numbers.


    Robert
    XSharp Development Team
    The Netherlands

    Please Log in or Create an account to join the conversation.

    Array with float 08 Dec 2022 10:14 #24700

    • Chris
    • Chris's Avatar


  • Posts: 3736
  • Hi Horst,

    Yes, the sample is great!

    The problem is that floating point data types like REAL and FLOAT cannot represent the decimal part of non-integer numbers precisely (that;s for all computer programming languages), there's always some precision lost which cascades when you do arithmetic operations on them. In your particular sample, the fTotal var ends up having a value of 19106.990000000027 and this is why the comparison with 19106.99 fails.

    In order to workaround this, VO introduced (and X# also supports it) the function SetFloatDelta(), which you can use to tell the runtime that it should treat floating point numbers which are different by just a tiny amount as equal. So, in your sample, if you use something like SetFloatDelta(0.000001), then it will work as you expect it.

    In .Net, there's a new data type exactly for that, System.Decimal, which is slower than REAL/FLOAT, but does not suffer from this lack of precision. If you change the data type of your locals fTotal and fKontoSaldo from FLOAT to Decimal, you will again get the expected results, without needing to also use SetFloatDelta().

    .
    XSharp Development Team
    chris(at)xsharp.eu

    Please Log in or Create an account to join the conversation.

    Array with float 08 Dec 2022 10:55 #24701

    • Horst
    • Horst's Avatar
    • Topic Author


  • Posts: 284
  • Hello
    Thanks Robert and Chris for explaining that.
    I will now only use Decimal instead of Float, because i will forget that float is making this little misstake :-)

    btw: when i am importing some excel data, there also has numbers in the file (on disc) like 15.12200000009 . But the input of the user was 15.122.

    Horst

    Please Log in or Create an account to join the conversation.

    Array with float 08 Dec 2022 11:04 #24703

    • Chris
    • Chris's Avatar


  • Posts: 3736
  • Hi Horst,

    Same issue, the value 15.122 cannot be represented exactly in the binary notation that FLOAT/REAL uses. It's a bit similar to that the result of the division 10/3 cannot be represented exactly in decimal notation, binary notation has similar problems as well.
    XSharp Development Team
    chris(at)xsharp.eu

    Please Log in or Create an account to join the conversation.

    • Page:
    • 1