consumer producer c# implementation with 1 timed consumer for a bulk sending algorithm

I 开发者_开发问答need to implement a module that can have multiple inputs to a dictionary (multiple threads writing to a dictionary) and 1 timed consumer that takes this dictionary, sends it away using some ISender and clears the dictionary for a new bulk of data. the problem is that i need to design my interlocks that way that the consuming thread takes the quickest snapshot of the bulk while allowing the producing threads to keep writing to a new cleared dictionary. what is the best consumer producer design you would suggest using interlocks and ConcurrentDictionary?

Best Regards!

Don't let the producer threads put the data in the dictionary directly. Let them put it in some thread-safe queue, such as BlockingCollection. Your consumer thread can then take items from the queue, build the dictionary and send it away, all without blocking the producer threads.

Essentially the same work gets done, but is "spread around" in a way that avoids most of the blocking.

If you are extra-worried about contention on that single queue, you can even have a separate BlockingCollection per producer thread and then use BlockingCollection.TakeFromAny in the consumer.

The problem is, of course, if your consumer threads do anything other than simply writing to the dictionary. If they need, for example, to check if the given key already exists in the dictionary, then this design suddenly becomes much more complicated.

The fastest way I can think of is to use multiple dictionary objects.

When your consumer thread runs, it creates a new ConcurrentDictionary and sets it as the "live" dictionary. This is fast and means the producers can carry on with minimal interruption.

The consumer thread now "owns" the previous dictionary object and can process its contents in its own time.