reading a large logfile

[Martel]

Hi, I'm just starting with VB .net and I am trying to write an application to read through a logfile. This logfile consists of thousands of lines, where each line represents one message.

 

My question is: what is the best way to read this file?

 

My intentions are to display the logfile on the screen with certain sections of certain messages highlighted (decided by user input) and also graph the value of certain parts of the message throughout the logfile.

 

I got the application to work in Excel using VBA but as a learning exercise and also to make it more useful I would like to do it in VB. I have been thinking about creating a very large array full of LogMessage objects but I am no longer convinced this is the best way.

[Hughng]

I guess it depends on what you try to do with each line?

- Are you going to parse the whole log into small section and do something else with it?

- Are you going to go back and use the data somewhere else later?

 

If you are going to use the log record more than few time and have to do a lot of stuff with it. Then my best guess is read them as array of string with each line is 1 string index of the array.

That way, if your log is really big the array still be able to handle the log file.

[Martel]

I thought it would be easier to process the messages once when the file is loaded. If they were strings I would have to read the relevant data each time I wanted to do some analysis later. Therefore I would have thought storing each message as an object rather than a string would be a better design?

 

These log files might have 60,000 messages which would result in an array of length 60,000, is this size acceptable or is there a better way to hold so many message?

 

Most of the time I will have to sequentially search through each msg btw

[mutant]

If you want to keep each line separate from the the other there is really no other way then having 60 000 array members. I would suggest using an ArrayList so you wouldnt have to declare your array with 60000 members right away, because there might be a chance there will be less lines to read.

[Martel]

How does an array list work? I've never used them before

[dynamic_sysop]

Private Arr As New ArrayList()

Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
   Dim strValue() As String = {"item1", "item2", "item3", "item4"}
   Arr.AddRange(strValue)
   MessageBox.Show(Arr(0) & " : " & Arr(1) & " : " & Arr(2) & " : " & Arr(3))

End Sub

there's a quick example.

[Martel]

Thanks,

congrats on the twins

[dynamic_sysop]

np and cheers :D