C# Socket.Receive message length

I'm currently in the process of developing a C# Socket server that can accept multiple connections from multiple client computers. The objective of the server is to allow clients to "subscribe" and "un-subscribe" from server events.

So far I've taken a jolly good look over here: http://msdn.microsoft.com/en-us/library/5w7b7x5f(v=VS.100).aspx and http://msdn.microsoft.com/en-us/library/fx6588te.aspx for ideas.

All the messages I send are encrypted, so I take the string message that I wish to send, convert it into a byte[] array and then encrypt the data before pre-pending the message length to the da开发者_高级运维ta and sending it out over the connection.

One thing that strikes me as an issue is this: on the receiving end it seems possible that Socket.EndReceive() (or the associated callback) could return when only half of the message has been received. Is there an easy way to ensure each message is received "complete" and only one message at a time?

EDIT: For example, I take it .NET / Windows sockets does not "wrap" the messages to ensure that a single message sent with Socket.Send() is received in one Socket.Receive() call? Or does it?

My implementation so far:

private void StartListening()
{
    IPHostEntry ipHostInfo = Dns.GetHostEntry(Dns.GetHostName());
    IPEndPoint localEP = new IPEndPoint(ipHostInfo.AddressList[0], Constants.PortNumber);

    Socket listener = new Socket(localEP.Address.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
    listener.Bind(localEP);
    listener.Listen(10);

    while (true)
    {
        // Reset the event.
        this.listenAllDone.Reset();

        // Begin waiting for a connection
        listener.BeginAccept(new AsyncCallback(this.AcceptCallback), listener);

        // Wait for the event.
        this.listenAllDone.WaitOne();
    }
}

private void AcceptCallback(IAsyncResult ar)
{
    // Get the socket that handles the client request.
    Socket listener = (Socket) ar.AsyncState;
    Socket handler = listener.EndAccept(ar);

    // Signal the main thread to continue.
    this.listenAllDone.Set();

    // Accept the incoming connection and save a reference to the new Socket in the client data.
    CClient client = new CClient();
    client.Socket = handler;

    lock (this.clientList)
    {
        this.clientList.Add(client);
    }

    while (true)
    {
        this.readAllDone.Reset();

        // Begin waiting on data from the client.
        handler.BeginReceive(client.DataBuffer, 0, client.DataBuffer.Length, 0, new AsyncCallback(this.ReadCallback), client);

        this.readAllDone.WaitOne();
    }
}

private void ReadCallback(IAsyncResult asyn)
{
    CClient theClient = (CClient)asyn.AsyncState;

    // End the receive and get the number of bytes read.
    int iRx = theClient.Socket.EndReceive(asyn);
    if (iRx != 0)
    {
        // Data was read from the socket.
        // So save the data 
        byte[] recievedMsg = new byte[iRx];
        Array.Copy(theClient.DataBuffer, recievedMsg, iRx);

        this.readAllDone.Set();

        // Decode the message recieved and act accordingly.
        theClient.DecodeAndProcessMessage(recievedMsg);

        // Go back to waiting for data.
        this.WaitForData(theClient);
    }         
}

Yes, it is possible you'll have only part of message per one receiving, also it can be even worse during transfer only part of message will be sent. Usually you can see that during bad network conditions or under heavy network load.

To be clear on network level TCP guaranteed to transfer your data in specified order but it not guaranteed that portions of data will be same as you sent. There are many reasons for that software (take a look to Nagle's algorithm for example), hardware (different routers in trace), OS implementation, so in general you should never assume what part of data already transferred or received.

Sorry for long introduction, below some advices:

1. Try to use relatevely "new" API for high-performance socket server, here samples Networking Samples for .NET v4.0

2. Do not assume you always send full packet. Socket.EndSend() returns number of bytes actually scheduled to send, it can be even 1-2 bytes under heavy network load. So you have to implement resend rest part of buffer when it required.

   There is warning on MSDN:

   There is no guarantee that the data you send will appear on the network immediately. To increase network efficiency, the underlying system may delay transmission until a significant amount of outgoing data is collected. A successful completion of the BeginSend method means that the underlying system has had room to buffer your data for a network send.

3. Do not assume you always receive full packet. Join received data in some kind of buffer and analyze it when it have enough data.

4. Usually, for binary protocols, I add field to indicate how much data incoming, field with message type (or you can use fixed length per message type (generally not good, e.g. versioning problem)), version field (where applicable) and add CRC-field to end of message.

5. It not really required to read, a bit old and applies directly to Winsock but maybe worth to study: Winsock Programmer's FAQ

6. Take a look to ProtocolBuffers, it worth to learn: http://code.google.com/p/protobuf-csharp-port/, http://code.google.com/p/protobuf-net/

Hope it helps.

P.S. Sadly sample on MSDN you refer in question effectively ruin async paradigm as stated in other answers.

Your code is very wrong. Doing loops like that defeats the purpose of asynchronous programming. Async IO is used to not block the thread but let them continue doing other work. By looping like that, you are blocking the thread.

void StartListening()
{
    _listener.BeginAccept(OnAccept, null);
}

void OnAccept(IAsyncResult res)
{
    var clientSocket = listener.EndAccept(res);

    //begin accepting again
    _listener.BeginAccept(OnAccept, null);

   clientSocket.BeginReceive(xxxxxx, OnRead, clientSocket);
}

void OnReceive(IAsyncResult res)
{
    var socket = (Socket)res.Asyncstate;

    var bytesRead = socket.EndReceive(res);
    socket.BeginReceive(xxxxx, OnReceive, socket);

    //handle buffer here.
}

Note that I've removed all error handling to make the code cleaner. That code do not block any thread and is therefore much more effecient. I would break the code up in two classes: the server handling code and the client handling code. It makes it easier to maintain and extend.

Next thing to understand is that TCP is a stream protocol. It do not guarentee that a message arrives in one Receive. Therefore you must know either how large a message is or when it ends.

The first solution is to prefix each message with an header which you parse first and then continue reading until you get the complete body/message.

The second solution is to put some control character at the end of each message and continue reading until the control character is read. Keep in mind that you should encode that character if it can exist in the actual message.

You need to send fixed length messages or include in the header the length of the message. Try to have something that allows you to clearly identify the start of a packet.

I found 2 very useful links:

http://vadmyst.blogspot.com/2008/03/part-2-how-to-transfer-fixed-sized-data.html

C# Async TCP sockets: Handling buffer size and huge transfers