Stange substraction operation

[rekam]

Hello !
I have a very strange problem. Look at this :

Dim a, b, c as double
a = 1
b = 0.96

c = a - b
msgbox(c)

Why does the msgbox display 0.0399999999999999999 ?????? The result of this simple operation shouldn't be just 0.04 ?

Really, I don't understand. And the worst : when b < 0.95, c is correct

Well, has anybody already had such a problem ?
Thanks!

 

[BlueOysterCult]

Double has something to do with a 64 bit floating point
Pi as Double is 3.1415926535

I think maybe use Single?

 

[rekam]

It's strange, but it's the same problem...I think I'll cope that with Math.Round or something...even if it's not, let's say, "professional"

 

[Mehyar]

I tried your example and my msgbox displayed 0.04, although if you debug the debugger will show 0.0400000000036.

I agree its weird, but I do not know why your msgbox didnt display 0.04 ....

 

[PlausiblyDamp]

If you want exact numerics use the Decimal data type. Double and Single are both floating point numbers and as such are prone to rounding errors.

 

[donnacha]

Yeah, Decimal.Add/Subtract/Multiply combined with Decimal.Todouble should help.

e.g.
Dim a,b,c as double

b = 1234.555
c = 0234.554
a = Decimal.ToDouble(Decimal.Subtract(b,c))

should give a double of 1000.001