Knowing when to stop reading from .NET socket on HTTP level

Both .NET Socket and NetworkStream offer Read method, which blocks, when there is no data available. The problem is that, if client doesn't know the size of the message, there is no reliable way of understanding when server has finished sending the message.

My question is - what principle do higher level libraries (like HTTP-level WebRequest) use in order to stop reading from the socket and present full message to the client, without resorting to any sort of readi开发者_运维百科ng timeouts? It seems like HTTP protocol doesn't offer any "EOF" tokens...

HTTP 1.1 manages this by using one of two mechanisms. Either the content-length is sent in the headers, or a chunked transfer encoding is used. The chunked transer encoding is one of the big advantages of HTTP 1.1 over HTTP 1.0 where, in the case that no content-length was sent, end of transmission could not be reliably detected resulting in abiguity between transmission problems and the actual end of the payload. In some cases (such as audio streams over HTTP) there is no content-length or other encoding that might indicate end of transmission, but in this case it would be of little importance to the client.

There are some ways to check if sockets have data pending. One is to simply check the Pending property of each socket, and only call Receive if there is data. However, this doesn't scale very well at all, and I would only recommend using it in apps with a single connection or so.

The Socket.Select function can take an IList of Sockets, and after a short period, will reduce this list to contain only sockets which have data pending. You can then call receive on them safely without blocking.

The other method, which is preferred, is to use the asynchronous socket API. (Call to BeginReceive, or BeginRead on the NetworkStream). In these cases, the call will not block, and you can continue executing code after the call. You pass a function as an argument which is to be executed when data is pending on the socket. This is handled in a background thread in the .NET threadpool, so there are concurrency issues involved here, unlike the former two methods. The async stuff is done at a much lower level in the OS, by using interrupts to tell the CPU when data is waiting on the network, rather than polling for data repeatedly like Select does.

In HTTP, content-length HTTP header defines when the socket can be closed.

In TCP, this is arbitrary and usually client or server sends a special message to notify that connection is being terminated.