Multi-threading jobs that need to maintain order

I have a c/c++ process that has a long queue and开发者_运维百科 every element in this queue needs to be sent to a multiple (TCP) servers. The single thread is an option that works, however it is slow.

I need to implement a multithreaded solution. My process does not know in advance the number of servers. The first idea is to create one thread for every server. A manager thread reads the new element in the queue finds the destination and dispatches to the thread that matches the destination.

One important note. Jobs might be dependent. I don't want to executed job_10 if job_5 is not completed. Order needs to be maintained.

First of all I would like your opinion on the issue. Second I am searching for C++ implementations for reference. Third I am looking for books/sources that describe similar issues.

Have you looked at boost asio? It supports the concept of a "thread pool", so rather than a single thread per server (that is synchronous), you can have a pool of threads handling a large number of connections asynchronously. Performance isn't too bad either...

How did you arrive at the conclusion that you need a multithreaded solution? What do you mean when you say that your single-threaded solution is "slow"? Do you mean that other processing (perhaps UI stuff?) is held up while the items are sent to the server? Or does the whole process just take too long?

Your description is ambiguous on this question: does each item need to be sent to all servers, or just to one? Seems like just one, but I'm not sure. If it must be sent to all, then the complexity of a multi-threaded solution increases greatly.

Can you remove an item from the queue once it's been sent, or do you need to wait for an acknowledgement from the server? If you must wait then your queue management will become tricky - you can't remove the item until it's been acknowledged, and so you'll have to mark it as pending somehow, and the thread that picks the next item from the queue will need to be sure it doesn't pick a pending item. And when the item has been sent successfully it will have to be removed from the queue, and this implies concurrent write access to the queue.

How do you handle errors? If an item can't be sent does it stay in the queue for later retry, or is it moved to a failed list, or is it just logged and discarded? In a multithreaded system you'll learn of the failure asynchronously, and that implies concurrent write access to the error handling procedure.

You say that your process doesn't know in advance how many servers there are. Can this number change during processing?

By all means look into multithreading, but you may find that your "slow" single-threaded solution serves your business needs best - easy to implement, and reliable. If not, then hopefully some of these questions will be helpful to you.