Marshalling data from unmanaged code

[jlkinchantilly]

I have a VB6 application that calls a method from a custom DLL that I think was written in C (or C++). Unfortunately, I don't have the original source code, so I cannot examine the method signature for date types etc.

 

Here is what worked in VB6:

Public Declare Function xyz_function Lib "xyz.dll" _

(ByVal x As Integer, ByVal y As Integer, refArray() As Byte) As Integer

 

Dim refData(0 To 31) As Byte

If xyz_function(CInt(0), CInt(2), refData()) = 0 Then

...

End If

 

 

After reading several articles, here is what I tried in VB.NET:

<DllImport("xyz.dll")> _

Public Shared Function xyz_function(ByVal x As Integer, ByVal y As Integer, _

ByVal refArray As IntPtr) As Integer

End Function

 

Dim refData(31) As Byte

refData.Initialize()

Dim dataLength As Integer = refData.Length + 1

Dim unmanagedMem As IntPtr = Marshal.AllocHGlobal(dataLength)

Marshal.Copy(refData, 0, unmanagedMem, dataLength)

If xyz_function(CShort(0), CShort(2), unmanagedMem) = 0 Then

Marshal.Copy(unmanagedMem, refData, 0, dataLength)

Marshal.FreeHGlobal(unmanagedMem)

...

End If

 

I don't receive any exceptions, however at the end of the operation, all slots of refData are zero. Actually after trying several different things, I discovered that if I replace refData.Initialize() with pre-initialization of the slots to random values, those values are what I get back after calling the external method. So evidently, the external method is not changing the values in the memory location.

 

My question is: Is there something obviously wrong with this code that would make it not work? Does anyone have a suggestion for how to get it to work keeping in mind that I have no way of obtaining the original source?

[PlausiblyDamp]

Try

Public Declare Function xyz_function Lib "xyz.dll" _
(ByVal x As Short, ByVal y As Short, refArray() As Byte) As Short

[jlkinchantilly]

I did change the Integers to Short - strange that the vb6 to .NET converter did not catch that - but it produced no change in the output from the dll.

[PlausiblyDamp]

After you changed the declare are you still using the Marshall class to pass information in and out? If so just try using an array of bytes like in the original VB6 version.

[jlkinchantilly]

That definitely does not work. The array of bytes comes back as one byte.

[PlausiblyDamp]

Try the following for a declaration

Public Declare Function xyz_function Lib "xyz.dll" (ByVal x As Short, ByVal y As Short, ByRef refArray() As Byte) As Short

and see if using a method body of

Dim refData(31) As Byte

If xyz_function(CShort(0), CShort(2), refData) = 0 Then
...
End If

works