File Access - Say What?

[tonofsteel]

My last post started as a question to get my code working, but in the end it is VERY slow. So now my question is: How does anyone create super fast file access. Like any text editing program you use today can load megabytes of file in fractions of a second. And then search out and erase words or complete lines throughout the whole file that match certain criteria within seconds.

My attempt at the basics of this, just loading a file takes a mere 10+ minutes. Or searching through a file erasing lines drags on for only god knows how long cuz i wasnt gonna wait for dat to finish.

So if anyone out there has even a shred, or a hint, or a direction at least for me to reasearch, it would be very helpful.

My current setup was a rich text box. I attempted two different methods, using the load file function that comes with the rich text box and also opening the file and reading it line by line (This would be beneficial because as it loads i can filter out lines i dont want)

So basically how do you: Have fast file access, search and delete fast in a text box, rich text box, etc. Do you use API? Assembly? Stop using Visual Basic .net?

 

[tonofsteel]

Well I finally got it, and anyone out there that plans on
processing a file with lots of data, such as a log from something,
and you want to filter out data, make sure you use a temp file, its
the fastest way I found to do it. The code I was using will give
you a general idea of the algorithm:

    Public Function PFilter(ByVal FilterIndex As String, ByVal FileName As String)

        Try
            ' Create an instance of StreamReader to read from a file.
            Dim sw As StreamWriter = New StreamWriter("C:\Temp.log")
            Dim sr As StreamReader = New StreamReader(FileName)
            '"I:\SSP\03 dwnld\07\20030717.All_Event.log"


            Dim line As String
            Dim IsItThere As Integer

            Dim count As Integer
            ' Read and display the lines from the file until the end 
            ' of the file is reached.
            Do

                line = sr.ReadLine()

                For count = 1 To FilterStruc(FilterIndex).NumberofFilterItems
                    IsItThere = InStr(line, FilterStruc(FilterIndex).FilterItems(count), CompareMethod.Binary)
                    If IsItThere > 0 Then Exit For
                Next count

                If IsItThere > 0 Then
                    sw.WriteLine(line)
                End If

            Loop Until line Is Nothing

            sr.Close()
            sw.Close()

        Catch Ex As Exception
            ' Let the user know what went wrong.
            MessageBox.Show(Ex.Source & Ex.Message)
        End Try

        DoWhileNotDone = False
    End Function

After this function executes, the file is then loaded by the
program into a rich text box. A 1.4 MB file to filter out about 10
terms, save them to a file, and load the file in the window takes
about 1 second.