beyerch Posted September 28, 2005 Posted September 28, 2005 Have a bit of a dilemma and I'm wondering what type of opinoins are out there ... I have a piece of equipment that I communicate with via a serial port. Data comes in at a pretty high rate of speed and the data is in "packets" of varying sizes. I currently use the mscomm control as VB.Net 2002/2003 does not have built in comm functionality yet. The data comes into the mscomm control and I have a thread that is waiting for for data to be in the input buffer. Once the thread sees data there, it grabs it and moves it out of the mscomm buffer area. Sample incoming data looks like ... ------------ RX : 08 05 04 03 02 01 02 03 04 18 Where .... - 1st byte on left is the "header" which states how many bytes are in the packet. (08) - 1st byte on the right is the checksum which is a summation of the bytes that are neither header or checksum... (18) I have no way to know WHEN a packet is starting (as no lines are set like DTR or anything). All I can do is ASSume when a packet would beging and then just do a checksum calc on it and if I don't get a good calc, drop the data; however, I haven't implemented that much effort yet. Currently every byte goes into a byte array. In order to match up a response from the hardware, I have a function that you give a expected response and a count of characters to return such as : Search : FF 05 04 03 , 10 Where : FF = Match ANY byte 05 = 2nd byte of return data is 5 04 = 3rd byte of return data is 4 03 = 4th byte of return data is 3 10 = Return 10 characters of data starting at the location matched ... Here's the inefficient part ..... In order for this to work, I have to do a pretty long and tedious match and iterate through the byte array (which could be a couple thousand characters depending on what is happening).... Once it finds a matching "sub string" it returns it back to the caller. Also, when a match is found, we need to remove it from the buffer of characters since we don't want to find one response twice ..... So after the search is completed, I also then have to delete those characters which is basically an array copy minus the range of bytes we just matched out.... The problems I am having are : - Speed Inefficiency: Goes up a lot with amount of bytes returned from hardware - Memory Useage : since its just a never ending byte array, it can get hefty if we have a lot of data coming in. What I was thinking about doing is instead of the bytArray switch this over to a circular stack type implementation. i.e. define an array of bytarrays (say 50). Each of the 50 would be a pointer to a bytArray. That bytArray would contain on "packet" of data. As data comes in , you just increment the last record pointer. The issue that comes to mind is that I may need to get data out in a different order than it came in ... for instance lets say we have 3 packets in the circular stack. IF i need the middle packet, it would screw up the begin and end pointers as the data would no longer be together. I guess you could do a linked list to solve this problem; however, then everytime you add a node you need to update the links and the same when removing. Also, how can you readily do something like this in vb.net as I don't think you can just do a malloc ? I also still have the dilemma of how do i determine when data come in is a valid packet ? The packet structure i have is pretty weak and without actually trying to construct everything into a packet and then validate the checksum, it will be hard to make each one into a packet .... Has anyone here had to do an app with a lot of serial data acquisition ? I'd really love to talk about what would be the "best case" way to handle this. Thanks ! Quote
bri189a Posted September 28, 2005 Posted September 28, 2005 Short on time so couldn't read the whole post but if your looking for something that can parse and find things in a long string really FAST look at Regular Expressions (a.k.a. REGEX)... I'm not an expert on them, but for finding 'matches' or 'patterns' it's unbelievably quick. Quote
kejpa Posted September 29, 2005 Posted September 29, 2005 Hi, if you have any control at all over the hardware sending the data put in delimiters and most of your problems are solved. VB2005 supports Serialports, maybe you should switch environment, and if you can't there is some free code manageing serialports for VS, quite good and customizeable. It seems to me like it's quite easy to get the information as soon as you know where to start reading, 1st byte indicates number of databytes. If it's not too heavy sending you can assume that the string you get from MSComm is the full length message. Transmissions with that little data is fairly quick and unless it's a flooding stream of byte jumping at you as soon as you open the communication ports there will be a short break between the messages. You can use that information to assume that there soon will come a start of a new message. HTH /Kejpa Quote
beyerch Posted September 29, 2005 Author Posted September 29, 2005 Hi, if you have any control at all over the hardware sending the data put in delimiters and most of your problems are solved. VB2005 supports Serialports, maybe you should switch environment, and if you can't there is some free code manageing serialports for VS, quite good and customizeable. It seems to me like it's quite easy to get the information as soon as you know where to start reading, 1st byte indicates number of databytes. If it's not too heavy sending you can assume that the string you get from MSComm is the full length message. Transmissions with that little data is fairly quick and unless it's a flooding stream of byte jumping at you as soon as you open the communication ports there will be a short break between the messages. You can use that information to assume that there soon will come a start of a new message. HTH /Kejpa Hi, Thanks for replying ! #1 - 2005 VB is out ? Sorry, I haven't been keeping up with current events lately. But given windows 95 came out in what 98, etc, etc, I didn't expect to see MS anything 2005 until 2007 :) I knew that release would have built in serial handling and I think that is the first thing I should do is upgrade. I don't like using the mscomm control from vb6 days. Its worked pretty well, but I'd rather have something with a bit more control. #2 - I do not have control over the hardware. You are 100% correct; however, that if they could toggle a line at the start of the message that would solve a helluva lot of my problems. That is not going to happen though. :( #3 - The data CAN move very quickly and I cannot afford to assume there will be a break and each one will be a copmlete message. #4 - The format of the data being returned to me, quite frankly sucks ... The first byte is xy where : x = 0 - 9, A-F, X-Z y = the length (hexadecimal) So basically almost any character received "could" be the first character of a block ..... I would have at least preferred some arbitrary byte (or 2 bytes) to come back first so I knew the communication was from the hardware but once again don't see that happening ..... The only way I could be sure I had a compelte packet would be to : (pseudo code) do while comm.inbuffer > 2 <--- Smallest packet is 3 bytes bytHeader = comm.readbyte(1) if comm.inbuffer >= HeaderLength(bytHeader) then if CalcCSUM = PacketCSUM then 'Store data as a record 'remove data from buffer else 'bad data remove from buffer end if end if loop ^^ Thats not 100% what to do, but to give an idea basically going to have to iterate through incoming data and just blindly take a byte and ASSume its a header byte and then try to construct a packet out of it. If it fails, move to the next byte (or erase all bytes since data error ??) and continue scanning ........ The catch for this is knowing what is a header byte ..... I have no way of 100% knowing and that really isn't helping me out here .... Lets forget that for a minute, any opinions on the most optimal way to store the data and search at a later date when I need to start finding those packets that we grab ? I'm trying to cut down on the time as for older machines iterating through a couple thousand byte byte array hurt performance .... :) :( Quote
Administrators PlausiblyDamp Posted September 29, 2005 Administrators Posted September 29, 2005 VB 2005 isn;t out yet although only a month or 2 till the proposed date so it could still be a viable option for you. The SerialPort class in 2005 offers a BaseStream property that you could use in conjunction with a BinaryReader object and let .Net take care of the buffering etc for you. Quote Posting Guidelines FAQ Post Formatting Intellectuals solve problems; geniuses prevent them. -- Albert Einstein
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.