Data Type Conversions

H

Heaven

Newcomer
Joined
Apr 17, 2003
Messages
2
Location
Centralia Wa
I'm using visual studio.net pro, and this is occuring when I'm using VB

Hope some one has the answer to this one.
I just want to know the Why this happens.

Here's my code:

Dim l as Long
Dim S as Single

l = Long.MaxValue 'set the long variable to its maximum value
Debug.WriteLine(l) 'display the results to output (which are 922337203684775807)

s= Convert.ToString(l) 'convert the long to a string
s -=1000000000000 'subtract 1 trillion form the value
l= Convert.toInt64(s) 'convert the answer back to a long

Debug.WriteLine(l) 'display the results to ourput (which are 9223370937343148032)

'end of code

now as you can see you don't get what you would expect. a single should have just rounded off at 7 places and returned the answer of 9223371 + a lot of trailing zeros.

I've looked at the .net library and the only thing I can find is that the results would be "inaccurate" duh.... I am trying to find out why . any ideas??? i'm sure it has something to do with overflow and possibly PositiveInfinity.
 
if you set 'Option Strict On' you cannot do this...
Visual Basic: 
s -=1000000000000 'subtract 1 trillion form the value
 
You take a 64-bit integer value, convert it to a string, set a 32-bit equal to it (the string value) and then proceed to perform subtraction on a string value...

You might, just might, want to rethink that very poorly written code.
 
re; data conversion

I Know that it is poorly written code. as a matter of a fact.. it's not a good to convert from a long back to a single. but.... that's not what I'm trying to find out. I'm trying to find out what is happening to the result. why is it coming back as inaccurately as it is. I think (and I may be wrong) that it might have something to do with when the non integer number is converted to binary from base 10 and then back to decimal for display. i'm wondering if maybe the fact that some base 10 non integer numbers, when converted to binary act more like an irrational number and go to positive infinity (like as an example pi. or 0.2 ) if anyone knows what is happening with the numbers, I'd sure like to know (it's turned into a vendeta for me.)
 
One thing I don't understand is why you declared S as Single, then you Convert it to String.
Did you mean to declare S as String ?

Is so, then this will work...
Visual Basic: 
Dim l As Long
Dim S As String

l = Long.MaxValue
S = l.ToString
S = (Convert.ToInt64(S) - 1000000000000).ToString
l = Convert.ToInt64(S)

MessageBox.Show(Long.MaxValue & ControlChars.NewLine & l.ToString)
 
Back
Top