Changed Parallel.ForEach Behaviour?

I have a report building class that uses parallel processing to build a batch of 800 or so reports.

Until yesterday it was working fine with the following code:

Parallel.ForEach(reports, report => { this.Build(report); });

Note: for debug purposes I'm exceuting the code from a test written in a test project.

Yesterday, it started failing. A bit of digging with Resource Monitor showed that the thread count associated with qtagent32.exe (the test runner) was ever increasing, and eventually caused the process to fail. It looked like something had suddenly become blocking.

I was able to fix the issue with a small change to my code:

Parallel.ForEach(reports, new ParallelOptions { MaxDegreeOfParallelism = 4 }, report => { this.Build(report);开发者_Go百科 });

But I'm still left wondering what changed?

It only takes 1 element to get a little slower. If it really is "same code", ie the same reports, it may have been the database. A little more data can make a query go a little slower.

The TPL sits on top of the ThreadPool, and the ThreadPool will slowly add new Threads.

Supppose one of your Reports takes longer than the threshold (500 ms) then the TP will create an extra Thread. If your Reports compete for something (Disk I/O most likely) than that extra thread will make it more likely other reports also exceed the threshold. With an avalanche as the result.

The new Fx4 Threadpool is smarter than the previous one, but it still is a crude heuristic. Your MaxDegreeOfParallelism is the right solution.