开发者

Thread Index in PLINQ Query?

Is there way to pass the thread "index" of a PLINQ query into one of it's operators, like a Select?

The background is that I have a PLINQ Query that does some deserialization, the deserialization method uses a structure (problably an array) to pass data to another method:

      ParallelEnumerable.Range(0, nrObjects)
        .WithDegreeOfParallelism(8)
        .Select(i =>开发者_开发知识库 this.Deserialize(serializer, i, arrays[threadIndex]))
        .ToList();

(threadIndex is the new variable I'd like)

If I knew the thread index I could create 8 of these arrays upfront (even if all might not be used) and reuse them. The Deserialization method might be called millions of times so every small optimization counts..


Re the thread index; note that the degree of parallelism is (IIRC) only the maximum number of threads to use; it doesn't have to use that number. Relying on threadIndex in the way you describe would seem to mean that you might actually only access arrays[0]. So in short, no: I don't think so.

You can, however, get the item index; so if that is what you mean, simply:

.Select((value, itemIndex) => this.Deserialize(
     serializer, i, arrays[itemIndex])).ToList();

It sounds (comments) like the intent is for obtaining a working buffer; in which case, I would (and indeed, do) keep a few convenient buffers in a container (with synchronized access etc), i.e.

  • attempt to obtain an existing buffer from a container (synchronized)
  • if not found, create a new buffer
  • (do work)
  • return/add buffer to the container (synchronized)

This will generally be very fast (much faster than buffer-allocation per iteration)


You can create ConcurrentBag of integers and then take from it before deserealization use as index and then return after. Every thread will occupy one array element at a time.

Consider using Parallel class. It is more appropriate when it comes to arrays. There are limitations on indexed versions of Select if you use ParallelEnumerable (Not in your case, but be cautious)


I would recommend using Parallel.ForEach which offers an overload that allows you to allocate (and cleanup) locals that will be passed into one parallel instance of the loop at a time. I've used this approach for processing of large sets of image data where I require buffers for various steps of the process.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜