How to improve TCP/IP server scalability when restricted by application thread pool

I have a TCP/IP server written in C# .net which can have 10,000 connections at once easy. However, when a callback is received from a socket, it is dealt with by a new thread in the application thread pool. This means that the real concurrent communication limitation is down to the number of threads within the thread pool. For example if those 10,000 开发者_运维技巧connections all attempt to send data at the same time, the majority will have to wait whilst the thread pool runs through as fast as it can. Can anyone share their experience with high performance socket services and advise how a large corporation would go about ensuring the 10,000 connections can not only be connected at the same time, but can also communicate at the same time? Thanks

Don't process the packets inline in the callback. Do the absolute minimum work there, and then hand them off to a separate worker thread pool via a producer-consumer queue that (ideally) never blocks the producer threads, which are your socket listeners. BlockingCollection<T> may be useful here.

You have to be careful that the queue does not grow unbounded - if your consumers are a lot slower than producers, and the queue grows under normal load, you have a problem to which throttling the network receives is the obvious solution, despite its undesirability.

YOu make a thought mistake here. Regardless how many thread you have, data always has to wait unless you have one CPU CORE PER CONNECTION. Scalability is not having unlimited paralellism, but being ab le to handle a lot of conenctions and keep the cpu at full power.

The thread pool is perfectly sized for that. Once the CPU reaches full utilization, you can not do anything else anyway.

and advise how a large corporation would go about ensuring the 10,000 connections can not only be connected at the same time, but can also communicate at the same time?

MANY computers that have like a total of 500 processor cores. The trick is: what latency is acceptable. You dont need instant communication. You try to sovle that from the wrong end.