Rounding on Doubles

P

pendragon

Junior Contributor
Joined
Aug 20, 2003
Messages
212
Location
Cambridgeshire
I am a little confused as to how double type variables are handled and hope someone can explain to me what is happening. The problem breaks down to this

I have 3 doubles defined and there values are as follows

a = 52.77, b = 15.15 and c = 37.62

I first do a -= b and show the result for a in a message box, It equals 37.62

I then do a -= c and show the result for a in a message box, but instead of a equaling 0 it equals 7.105427357601e-15

If I do a = math.round(a - b,2) or change them from doubles to decimals then it is ok

I was just wondering why??
 
Are a and c EXACTLY the same value, or are you getting these values by rounding? Do they both contain exactly 2 decimal places? If they are not exactly the same (no matter how close) the Double's precision is high enough that it will not exactly equal zero (in your case, 0.00000000000000710547357601).
 
In theory yes both a and c are the same, at least when I look at them with a message box but as a has already had a calculation done on it then the binary fig. may not be.

As you might be able to tell I have been doing a bit more research on this and have spent half the day reading tech papers on the IEEE standard.

I think I agree with PlausiblyDamp and will use decimal variable types as I only need 4 decimal points.

Thank you for your replys
 
I'd seen this before but didn't worry too much as I was never interested in comparing the return values - I just wanted the results. But in doing some testing, I can't seem to figure out how to see the "real" value being stored after the calculation. Meaning, as far as I can tell, the result of a-b-c should be 0. Or more importantly, a-b is 37.62 - I can see no indication that the internal representation contains extra digits anywhere. It must be the dynamic nature of the double to "move" the decimal point and exponential value around internally, so that c="37.62" can be represented as 32.62 with an exp of 0 or .3267 with an exp of -2. I wonder if it's possible to "see" how it's being stored so that you can adjust it?

-Nerseus
 
Nerseus

I agree, a-b-c should equal 0, but it is during the a-b that things go wrong and can only assume that there must be more than one way for 37.62 to be represented in base 2 hex, just in the same way that 37.62 - 37 does not equal .62 but .619999999999997.

I did try to find a way of looking at how it was being stored but am no where near being an expect at vb.net yet so could find nothing.

If anybody out there does know of a way then I would love to know just to satisfy my curiosity of what is going on

Thanks
 
Hi folks,
isin't this annoying, you would think the system would be smart enough to avoid this. I had the same problem in doing additions. I wanted to check againt 100% and all the individual bits added up to 100 but the result was 100.000000000000001 so my comparison did not work.

I have found a solution which I find irritating to have to do, but you do what you need to do to get the thing to work even if it is daft.

The solution uses Decimal as a middle man to get around the rounding (pardon the pun). e.g.

dim res, a,b,c as double

a = 75.25
b = 24.74999
c = 00.00001

res = Decimal.ToDouble(Decimal.Add(a,Decimal.Add(b,c)))

The result should be 100.0

You should be able to solve yier problem with the Decimal.Subtract

Lets hope next year is less .NET bug ridden.....
 
Back
Top