Best approach to non blocking server/listening socket in a multi-thread application on Windows?

I'm writing a TCP server/client application on Windows, to become familiar with the Winsock API. I come from an UNIX background and would like to know which of these could be the best approach to implement the application:

First the specification

• Must scale well on multiprocessor and single-processor systems.
• No hardset limit of connections.
• Application can both listen for connections, acting as server, and act as client.
• Multi threaded.

First approach:

• Non-blocking select-like socket for listening, in the 'server' thread.
• for each client connecting we spawn a separate thread.

Second approach:

• Blocking socket for listening, in the 'server' thread.
• for each client connecting we spawn a separate thread.

Third approach:

• Non-blocking select-like socket for listening, in the 'server' thread.
• No separate thread for each incoming connection, the protocol would need state information kept across sessions I suppose.

I wonder wh开发者_JAVA百科at is the most efficient and scalable approach, and especially if it can work with a UDP socket too.

Note: I'm writing the application in plain and old C. No .NET nor C++ involved, C++ exceptions disabled too.

As Gary says, I/O Completion Ports are the most efficient way to manage multiple network connections in a non-blocking/async manner on Windows platforms.

With IOCP you get notified when your networking operations complete and you can process these completions with a small number of threads. You get to decide how many threads you allocate to process the completions and the kernel decides when to use the threads that you're providing. It uses them in a LIFO order, to reduce context switching, so that if you are only using the minimal number of threads required at any point and you're reusing the same threads rather than cycling through all of the threads that you have available for use.

The asynchronous nature of IOCP programming can be a little confusing to start with, but once you get the hang of it it's fairly straight forward.

I have some free IOCP server code which demonstrates the basics and provides some example servers that are pretty easy to build on. You can find the code here: http://www.serverframework.com/products---the-free-framework.html. That page also links to some articles that I wrote to explain the code.

Relating this to the detail of your question. You should be looking at a variation on your third approach. Use AcceptEx() to accept new connections, this can be used in an asynchronous manner and so you don't need a separate thread for connection acceptance and can use the threads that are also processing your overlapped/async read and write operations.

I've written an asynchronous client which does not use blocking sockets, so if you're interested in that approach, then take a look at my client: http://codesprout.blogspot.com/2011/04/asynchronous-http-client.html

It's an HTTP client, but I've shown very little HTTP protocol processing in there, it's all just .NET sockets. The server would work in a similar way: you can take advantage of the *Async methods such as AsseptAsync.

Under Windows, the best performances are achieved by using I/O completion calls.

This is because the lists and queuing mechanism is done in the kernel, far from the heavy user-mode overhead (which drags your code down if you dare to do the hard work yourself).

Unfortunately, Windows I/O completion calls need to allocate many threads to scale and this is quickly killing the performances (as compared to Linux epoll which can scale independently of the number of worker threads you decide to involve in the task).

Recently, I discovered http://gwan.com/ a Web server which came from Windows and was then ported under Linux. And their authors describe the problem in details on their forum.