Caching a network stream

My C# program is receiving a lot of data via multiple sockets and it appears that I have problems that the TCP cache on the Windows Kernel is getting overloaded.

This can be seen by the following results from the tcpdump from the server:

18:02:18.963370 IP xxxx > yyyy.zzz: . ack 10017011 win 0

Which data structure/strategy would you use to cache the read data to avoid the overload on C#?

I am currently working with a delegate for every socket in a seperate thread which is called after every read record, but this seems to be a problem, so that I might need to split it into two threads.

Or should I look at tweaking the TCP parameters on the Windows side? I am using Windows 7开发者_StackOverflow中文版.

Are there any books for the kernel optimization?

I was really blindsided that the program got slower even though the load is around 20-25 % for the full machine and 50-60 % on the highest loaded core.

Without knowing too much about the architecture of your program, I would guess that your program itself is the bottleneck. If your application doesn't use a background worker to collect TCP messages and store them to be consumed by the code that does the "heavy lifting", then your program is reading the existing buffer, then processing what it has while the TCP buffer refills. This vicious cycle can only have one ending; eventually the buffer will overflow while waiting for your program to finish processing what it got last time.

I would look at refactoring your TCP reception into an ETL-type model. You need one thread whose job is to listen for TCP input, extract the data from the buffer and enqueue it in a collection that's visible to the rest of the program. It should do nothing else other than yank data out of the buffer and shove the raw messages into this queue, and it should do this as fast as it can.

With the received messages in memory, the rest of your program can take almost as long as it wants to consume each message. You still have to ensure you don't overload this in-memory queue, but as a single .NET object can reference up to 2GB worth of memory, you're much less likely to run into trouble. If you ARE overflowing this in-program queue, you need to look at either making the processing more efficient, or throwing more threads at this second stage of the process.