IO Completion Ports vs. Thread Pool API

I had a problem described here, and I was suggested to use IO completion ports or thread pool.

I have implemented IO completion, calling PostQueuedCompletionStatus to enqueue the task and GetQueuedCompletionStatus 开发者_开发技巧to get the next task to execute it. I am using IO completion port as a multi-producer/multi-consumer thread-safe FIFO container without explicit locks. This makes me have full control over the threads, because I may need to terminate the ones process for a long time and report them. Also GetQueuedCompletionStatus waits the calling thread if there is no task left.

Aside from the termination, thread pool suits my needs: my tasks are completed less than a millisecond, but there are lots of them. Also calling QueueUserWorkItem and letting the OS do the synchronizations and executions is simpler.

Are there any differences between two approaches performance-wise? Any comments about my implementation?

Completion Ports are designed to avoid unnecessary context switching. When your thread that is calling GetQueuedCompletionStatus is done processing a work item, it can call right back to GetQueuedCompletionStatus to continue to get more work within its current CPU timeslice.

@Jonathan - If you have blocking calls, then those should pretty much never be made on a thread pulling work items. They should either be performed async (with a Begin/End or *Async call) or block on another thread (worker thread pool). This will ensure that all of your threads servicing the completion port are actually doing work instead of wasting time blocking when other work items are available.

A slight clarification: if you are managing your own threads and calling GetQueuedCompletionStatus, then you have created your own completion port separate from the IO completion port and associated thread pool used by the OS for async IO calls.

IO Completion Port (IOCP) is typically used with a threadpool to handle IO events/activities while the WinAPI threadpool (which you indicate via QueueUserWorkItem) is merely Microsoft's implementation of a typical threadpool that will handle non-IO tasks.

Looking at your linked thread, it looks like you're simply dequeing a task from a FIFO list which has nothing to do with IO. So, the latter is most likely what you are after. I don't think performance difference should be your concern here as opposed to which is the right API for what you're doing.

EDIT: If you need full control over thread creation and termination (although it's never OK to terminate a thread as the stack will not unwind), then you'll be better off creating your own threadpool by using WaitForSingleObject (or rather MultipleObjects for exit signal as well) and SetEvent. WinAPI's threadpool is basically Microsoft's automatic thread creation and termination depending on the load of the threads.

If you use IO completion ports, and create your own X threads which call GetQueuedCompletionStatus(), and you have X tasks that takes a long time (say, read from network), then all threads would be busy and further requests will starve. AFAIU, thread pool will spin another thread in this case.

Also, never use TerminateThread()! When allocating memory from the heap, a thread temporarily acquire that heap's CRITICAL_SECTION. So if a thread is terminated in the middle of this, other threads trying to allocate from the same heap will hang. And, there's no way for you to know whether the thread is allocating or not.

The difference between QueueUserWorkItem and IOCompletionPorts is QueueUserWorkItem is an easy to use higher level abstraction. Its easy to see how QueueUserWorkItem could be implemented on top of IOCompletionPorts (I don't know that it is).

The practical difference is - QueueUserWorkItem creates (and manages) threads in the thread pool. As such, it doesn;t know up front how many threads it will need: It starts with a few( perhaps just one) and then creates additional pool threads at intervals if the thread pool has no free threads to handle queued items.

This means that using QueueUserWorkItem can result in a significant delay to process items if many items are added while the pool is small, consequently, bursts of items being added can cause the pool to grow larger than it needs to.