C# threading question

My existing c# app updates a sharepoint site collection. Now I've been task to make my c# app update not 1 but 15 site collections. It would take too long to run sequentially, so threading would be a good solution.

H开发者_开发百科owever I can't just open 15 threads at once, I would like to have a configurable value which will be the max number of threads to run at once.

What I would expect is:

• Define Max_Threads variable...
• Assign a queue of tasks (functions) at the thread pool, in this case 15 tasks or function calls.
• The thread pool then executes each thread, however is limited by how many open threads (5 for example).
• Once a thread completes, the threadpool then reuses that thread to do another item of work, until all the work is complete.

So my question is: Is all this build into .net threading. Or do I have to manually create the thread management code?

EDIT: this is a framework 3.5 project, sorry for not mentioning this before.

just use Tasks and don't think about the thread-numbers. The TPL will do all this for you.

The book C# in a Nutshell, 4.0, (by Albahari & Albahari) gives a neat example of simultaneously calling many "web clients" and downloading information. It uses PLINQ, does it in one or so lines of code. The PLINQ library handles the creation of the threads, etc. If you look at it, it doesn't tend to make too many threads, although you can force a limit of the maximum number of threads at a time. Similarly, you can use the Parallel.Foreach() and use the "ParallelOptions" parameter to limit the number of threads.

The nice thing is that you never have to create the threads yourself - it's automatic. And it does a fine job of load balancing.

A good tutorial is http://www.albahari.com/threading/ - look at Part 5 on Parallel Programming for lots of examples of using PLINQ and Parallel.Foreach.

Also, the Wagner book on Effective C# (4.0) has similar examples.

You can use Parallel framework that comes with .NET 4.0: http://msdn.microsoft.com/en-us/library/dd460693.aspx

Use TPL that will in turn use .NET thread pool (by default). It suppose to automatically adjust to the environment it runs in:

Beginning with the .NET Framework version 4, the default size of the thread pool for a process depends on several factors, such as the size of the virtual address space.

You only need to do your own thread management in a very limited number of cases:

• You require a foreground thread.

• You require a thread to have a particular priority.

• You have tasks that cause the thread to block for long periods of time. The thread pool has a maximum number of threads, so a large number of blocked thread pool threads might prevent tasks from starting.

• You need to place threads into a single-threaded apartment. All ThreadPool threads are in the multithreaded apartment.

• You need to have a stable identity associated with the thread, or to dedicate a thread to a task.

If you use .NET 3.5 you should look at the article that explains how to achieve some of TPL functionality using Thread pool directly: Does the TPL really enable you to do anything you couldn't do with ThreadPool in .NET 2.0?.

Consider using the Reactive Extensions backport for .NET 3.5. This will allow you to use the Task Parallel Library (TPL). If this is not an option then continue reading.

I would throw your entire site collection at the ThreadPool all at once and see how it works out before trying to throttle the work items.

If you do feel it is necessary to throttle the number of simultaneous work items in the ThreadPool then you can use a semaphore to throttle them. However, be careful not to block a ThreadPool thread as that is considered bad practice. Instead, block the queueing thread.

int pending = sites.Count;
var finished = new ManualResetEvent(false);
var semaphore = new Semaphore(5, 5);
foreach (string site in sites)
{
  semaphore.WaitOne();
  ThreadPool.QueueUserWorkItem(
    (state) =>
    {
      try
      {
        // Process your work item here.
      }
      finally
      {
        semaphore.Release();
        if (Interlocked.Decrement(ref pending) == 0)
        {
          finished.Set(); // This is the last work item.
        }
      }
    }, null);
}
finished.WaitOne(); // Wait for all work items to complete.