Running many simultaneous threads with Task.Factory doesn't work right

My application handles a data feed. When a new packet comes in, a dispatcher collects it, and raises an event that a proper listener can pick up and do what it needs to do.

I am trying to simulate the live feed to perform some testing. I made a class that feeds the dispatcher with packets, based on the number of active listeners.

This is the code I use to start the Feed() method which sits in memory and generate a packet every given interval:

 foreach (var item in Listeners)
            {
                object listener = item;

                Task.Factory.StartNew(()=> Feed(listener), TaskCreationOptions.LongRunning);

            }

The Feed() method works something like this:

while(run)
{
packet = GenerateThePacket(listener.Id); // Make a packet with the listener id
FeedHandler.OnPacketRecieved(this, packet); // Raise开发者_C百科s the FeedHandler's event as if it came from outside.

Thread.Sleep(1000/interval) // interval determines how many packets per second
}

So, if I have a 100 listeners, it'll start 100 instances of Feed(), each with different listener id, and fire up PacketRecieved events at the same time with the requested interval.

I guess many of you already know whats bad about it, but I'll explain the problem anyway: When I use an interval of 1 or 2 it works great. When I choose 10 (that is, a packet every 100ms) it doesn't work right. Each thread fires up at different intervals, where the latest one created works good and fast (10/sec), and the first ones create works real slow (1/sec or less).

I guess that a 100 threads can't operate at the same time and so they are just waiting. I think.

What exactly is happening and how can I implement a true feed generator which simulates 10 packets a sec simultaneously for a 100 listeners.

I think you're approaching this from the wrong angle....

Have a read through these:
http://blogs.msdn.com/b/pfxteam/archive/2010/04/21/9997559.aspx (link to pdf on there) http://www.sadev.co.za/content/pulled-apart-part-vii-plinq-not-easy-first-assumed

In a nutshell, the Task library will get a thread from the threadpool and if one is not available, the tasks will be queued until a thread is available..... so, the number of threads than can concurrently run depends on your system and the size of your threadpool.

For me, there are 2 ways to go... use the Parallel.ForEach static method or use the PLinq AsParallel() option as described in the articles above. At the end of the day, down to you which one to use.

Using plinq... something like this:

var parallelQuery = Listeners.AsParallel().Select(item=> Feed(item));   //creates the parallel query
parallelQuery.ForAll(item=> <dosomething>);     //begin the parallel process and do whatever you need to do for each result.

your feed method/object can look like this:

 while(run)
    {
    packet = GenerateThePacket(listener.Id); 
    FeedHandler.OnPacketRecieved(this, packet); // Raises the FeedHandler's event as if it came from outside.

       //No more Thread.Sleep
    }

This is just a basic intro for you but the links i've added above are quite helpful and informative. It's up to you which method to use. Keep in mind there are additional options you can add.... all in the links above.

Hope this helps!