Fast XML to Dataset code

[element]

Hi,

 

I am using the basic Data.DataSet functions to load/save an xml document using a FileStream to a DataSet and vice versa. However the file rapidly becomes very large, and is quite slow to load on startup.

 

I would like to know if it is possible to load the file in Asyncronously with some kind of callback so I could provide the user with feedback on the progress it has made loading the file into the DataSet. I am not sure how to go about doing this however.

 

This is all I am using to load the XML to the DataSet:

Dim xStream As New IO.FileStream(filename, IO.FileMode.Open)

m_Data.ReadXml(xStream)

 

Thanks in advance,

Tom

[dynamic_sysop]

have you tried creating a new thread? here's a quick example ( allowing your form to carry on loading as normal as the file gets read in to the app )...

Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
   Dim thread As New Threading.Thread(AddressOf testing)
   thread.Start()
End Sub

Private Sub testing()
   '/// loading a 6 meg file as a test in this case
   Dim path As String = Environment.GetFolderPath(Environment.SpecialFolder.Personal) & "\Win32api.txt"
   Dim sReader As New IO.StreamReader(New IO.FileStream(path, IO.FileMode.Open))
   While Not sReader.Peek
       ListBox1.Items.Add(sReader.ReadLine)
   End While
End Sub

[element]

Thank you very much your post has been very helpful :)

 

I've used Threading, and a Thread Delegate courtesy of: http://www.xtremedotnettalk.com/t70749.html

 

It now loads whilst everything else is loading, so other stuff can be done at the same time. The only problem is i can start using the database before it's fully loaded, but it seems to work anyway???

 

I shall be using this up until I can find a way of inserting the XML source to the dataset on the fly and display progress. Maybe a way of doing this could be to load the XML source in with an XMLDOM, write the schema to the dataset, and manually add each record...

 

Cheers, Tom

[Nerseus]

I've used the XmlDataDocument class to get an XmlDocument from a DataSet (they're kept in sync). I don't know if it will work the other way around though. If it does (you'll have to check), you might be able to load the file using the XmlDocument's load() method to load the file directly and then link it to a DataSet. Again, no idea if that will work, but might be worth investigating.

 

This assumes that loading an XmlDocument from a file is faster than loading a DataSet from the same file. If the file is extremely large, you're better of sticking with something else (threading for example, as you're trying now).

 

-ner

[element]

Nerseus,

 

That looks like it is worth investigating, I shall do so after work :)

 

Thanks for that tip, Tom