How to cancel waiting in select() on Windows

In my program there is one thread (receiving thread) that is responsible for receiving requests from a TCP socket and there are many threads (worker threads) that are responsible for processing the received requests. Once a request is processed I need to send an answer over TCP.

And here is a question. I would like to send TCP data in the same thread that I use for receiving data. This thread after receiving data usually waits for new data in select(). So once a worker thread finished processing a request and put an answer in the output queue it has to signal the receiving thread that there are data to send. The problem is that I don't know how to cancel waiting in select() in order to get out of waiting and to call send() .

Or shall I use another thread solely for sending data over TCP?

Updated

MSalters, Artyom thank you for you answers!

MSalters, having 开发者_如何转开发read your answer I found this site: Winsock 2 I/O Methods and read about WSAWaitForMultipleEvents(). My program in fact must work both on HP-UX and Windows I finally decided to use the approach that had been suggested by Artyom.

You need to use something similar to safe-pipe trick, but in your case you need to use a pair of connected TCP sockets.

1. Create a pair of sockets.
2. Add one to the select and wait on it as well
3. Notify by writing to other socket from other threads.
4. Select is immediately waken-up as one of the sockets is readable, reads all the data in this special socket and check all data in queues to send/recv

How to create pair of sockets under Windows?

inline void pair(SOCKET fds[2])
{
    struct sockaddr_in inaddr;
    struct sockaddr addr;
    SOCKET lst=::socket(AF_INET, SOCK_STREAM,IPPROTO_TCP);
    memset(&inaddr, 0, sizeof(inaddr));
    memset(&addr, 0, sizeof(addr));
    inaddr.sin_family = AF_INET;
    inaddr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
    inaddr.sin_port = 0;
    int yes=1;
    setsockopt(lst,SOL_SOCKET,SO_REUSEADDR,(char*)&yes,sizeof(yes));
    bind(lst,(struct sockaddr *)&inaddr,sizeof(inaddr));
    listen(lst,1);
    int len=sizeof(inaddr);
    getsockname(lst, &addr,&len);
    fds[0]=::socket(AF_INET, SOCK_STREAM,0);
    connect(fds[0],&addr,len);
    fds[1]=accept(lst,0,0);
    closesocket(lst);
}

Of course some checks should be added for return values.

select is not the native API for Windows. The native way is WSAWaitForMultipleEvents. If you use this to create an alertable wait, you can use QueueUserAPC to instruct the waiting thread to send data. (This might also mean you don't have to implement your own output queue)

See also this post: How to signal select() to return immediately?

For unix, use an anonymous pipe. For Windows: Unblocking can be achieved by adding a dummy (unbound) datagram socket to fd_set and then closing it. To make this thread safe, use QueueUserAPC:

The only way I found to make this multi-threadsafe is to close and recreate the socket in the same thread as the select statement is running. Of course this is difficult if the thread is blocking on the select. And then comes in the windows call QueueUserAPC. When windows is blocking in the select statement, the thread can handle Asynchronous Procedure Calls. You can schedule this from a different thread using QueueUserAPC. Windows interrupts the select, executes your function in the same thread, and continues with the select statement. You can now in your APC method close the socket and recreate it. Guaranteed thread safe and you will never loose a signal.

The typical model is for the worker to handle its own writing. Is there a reason why you want to send all the output-IO through selecting thread?

If you're sure of this model, you could have your workers send data back to the master thread using file descriptors as well (pipe(2)) and simply add those descriptors to your select() call.

And, if you're especially sure that you're not going to use pipes to send data back to your master process, the select call allows you to specify a timeout. You can busy-wait while checking your worker threads, and periodically call select to figure out which TCP sockets to read from.

Another quick&dirty solution is to add localhost sockets to the set. Now use those sockets as the inter-thread communication queues. Each worker thread simply sends something to its socket, which ends up on the corresponding socket in your receiving thread. This wakes up the select(), and your receiving thread can then echo the message on the appropriate outgoing socket.