C# 4.0 and queue with limited non-dedicated threadpool threads consumers

C# 4.0's new BlockingCollection doesn't answer a simple requirement we need:

* Concurrent-queue for tasks items.

* Consumers - limited N threads at a time - from the threadpool (threads which are not dedicated to this queue and are not blocked if there are no items in the queue). especially useful for resources usage we need to limit with only one (N = 1) thread at a time and the queue can be empty from time to time.

SmartThreadPool have a nice implementation for this purpose, using threads-group of the size of 1.

I was looking for a nice solution with the new C# 4.0 parallel lib, leveraging the .Net internal threadpool. I can think of a few solutions for thi开发者_运维知识库s, but I wonder if there's something elegant which I'm missing.

What do you think?

Regards, Shlomi

Tasks in the thread pool shouldn't block. While work has been done in .NET 4.0 to help support it, the Threadpool isn't really meant for long-running tasks and certainly not meant for operations that block (rule of thumb, if thread startup time is negligible to the time taken for the task then it's better for you to spin up your own thread).

For what it sounds like you're trying to do you might be better off with the IObservable stuff. Take a look at the Rx (Reactive) Extensions for some really cool things you can do with IObservable. By default, IObservable only handles one request at a time but you can easily take each object as it is sent to you and throw it on the threadpool (for actual processing) and/or use TPL. A TaskScheduler with max parallelism might help you here.

I can't think of anything out of the box that will do what you're looking for. You may want to pair a ConcurrentQueue with an EventWaitHandle and some number of worker threads that you can control.

if anyone interests in a proposed solution, please see the following thread under msdn forums: msdn parallel computing forums question.

It's question I asked relevant to the above and Stephen Toub gave me a satisfying answer.

Shlomi